-- Logs begin at Thu 2022-05-19 15:41:33 UTC, end at Thu 2022-05-19 17:10:28 UTC. -- May 19 16:27:09.273554 dv-cinder-842433-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 19 16:27:09.283276 dv-cinder-842433-1-iscsi devstack@keystone.service[80456]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 19 16:27:09.285265 dv-cinder-842433-1-iscsi devstack@keystone.service[80456]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu May 19 16:27:09 2022] *** May 19 16:27:09.285265 dv-cinder-842433-1-iscsi devstack@keystone.service[80456]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 19 16:27:09.285265 dv-cinder-842433-1-iscsi devstack@keystone.service[80456]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 19 16:27:09.285265 dv-cinder-842433-1-iscsi devstack@keystone.service[80456]: nodename: dv-cinder-842433-1-iscsi May 19 16:27:09.285265 dv-cinder-842433-1-iscsi devstack@keystone.service[80456]: machine: x86_64 May 19 16:27:09.285265 dv-cinder-842433-1-iscsi devstack@keystone.service[80456]: clock source: unix May 19 16:27:09.285265 dv-cinder-842433-1-iscsi devstack@keystone.service[80456]: pcre jit disabled May 19 16:27:09.285265 dv-cinder-842433-1-iscsi devstack@keystone.service[80456]: detected number of CPU cores: 4 May 19 16:27:09.285265 dv-cinder-842433-1-iscsi devstack@keystone.service[80456]: current working directory: / May 19 16:27:09.285265 dv-cinder-842433-1-iscsi devstack@keystone.service[80456]: detected binary path: /usr/bin/uwsgi-core May 19 16:27:09.285265 dv-cinder-842433-1-iscsi devstack@keystone.service[80456]: your processes number limit is 47680 May 19 16:27:09.285265 dv-cinder-842433-1-iscsi devstack@keystone.service[80456]: your memory page size is 4096 bytes May 19 16:27:09.285265 dv-cinder-842433-1-iscsi devstack@keystone.service[80456]: detected max file descriptor number: 2048 May 19 16:27:09.285265 dv-cinder-842433-1-iscsi devstack@keystone.service[80456]: lock engine: pthread robust mutexes May 19 16:27:09.286092 dv-cinder-842433-1-iscsi devstack@keystone.service[80456]: thunder lock: enabled May 19 16:27:09.286092 dv-cinder-842433-1-iscsi devstack@keystone.service[80456]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 19 16:27:09.286092 dv-cinder-842433-1-iscsi devstack@keystone.service[80456]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 19 16:27:09.330503 dv-cinder-842433-1-iscsi devstack@keystone.service[80456]: Python main interpreter initialized at 0x5643111049c0 May 19 16:27:09.330503 dv-cinder-842433-1-iscsi devstack@keystone.service[80456]: python threads support enabled May 19 16:27:09.330503 dv-cinder-842433-1-iscsi devstack@keystone.service[80456]: your server socket listen backlog is limited to 100 connections May 19 16:27:09.330503 dv-cinder-842433-1-iscsi devstack@keystone.service[80456]: your mercy for graceful operations on workers is 90 seconds May 19 16:27:09.330926 dv-cinder-842433-1-iscsi devstack@keystone.service[80456]: mapped 403077 bytes (393 KB) for 2 cores May 19 16:27:09.330926 dv-cinder-842433-1-iscsi devstack@keystone.service[80456]: *** Operational MODE: preforking *** May 19 16:27:09.331171 dv-cinder-842433-1-iscsi devstack@keystone.service[80456]: *** uWSGI is running in multiple interpreter mode *** May 19 16:27:09.331171 dv-cinder-842433-1-iscsi devstack@keystone.service[80456]: spawned uWSGI master process (pid: 80456) May 19 16:27:09.331382 dv-cinder-842433-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 19 16:27:09.332514 dv-cinder-842433-1-iscsi devstack@keystone.service[80456]: spawned uWSGI worker 1 (pid: 80466, cores: 1) May 19 16:27:09.332514 dv-cinder-842433-1-iscsi devstack@keystone.service[80456]: spawned uWSGI worker 2 (pid: 80467, cores: 1) May 19 16:27:09.332514 dv-cinder-842433-1-iscsi devstack@keystone.service[80456]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 19 16:27:15.074324 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 16:27:15.074324 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: from cryptography.utils import int_from_bytes May 19 16:27:15.075098 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 16:27:15.075098 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: from cryptography.utils import int_from_bytes May 19 16:27:15.076584 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 16:27:15.076584 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: from cryptography.utils import int_from_bytes May 19 16:27:15.077044 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 16:27:15.077044 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: from cryptography.utils import int_from_bytes May 19 16:27:15.125658 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 16:27:15.125870 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 16:27:15.126032 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] command line args: [] {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 16:27:15.126318 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 16:27:15.126408 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 16:27:15.126487 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ================================================================================ {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 16:27:15.126564 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 16:27:15.126737 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] command line args: [] {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 16:27:15.126820 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] admin_token = **** {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.126983 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] config_dir = [] {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.127070 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 16:27:15.127174 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ================================================================================ {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 16:27:15.127252 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.127357 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] config_source = [] {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.127435 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] admin_token = **** {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.127540 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] debug = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.127662 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] config_dir = [] {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.127739 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.127978 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.128048 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] config_source = [] {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.128114 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.128178 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] debug = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.128445 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.128542 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.128675 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.128779 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.128874 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.128974 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.129069 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] list_limit = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.129204 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.129313 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] log_config_append = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.129418 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.129513 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.129613 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] list_limit = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.129707 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] log_dir = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.129800 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] log_config_append = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.129898 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] log_file = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.130006 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.130092 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.130214 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] log_dir = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.130294 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.130410 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] log_file = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.130410 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.130593 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.130668 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.130792 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.130792 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.130980 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.131053 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.131178 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.131263 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.131335 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.131456 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.131536 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.131656 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.131735 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.131901 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.131980 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.132135 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.132213 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.132367 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.132446 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.132600 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.132674 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.132834 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.132916 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.133082 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.133190 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.133351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.133630 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.36/identity {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.133715 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.133819 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] publish_errors = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.134054 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.134135 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.36/identity {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.134284 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] publish_errors = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.134366 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.134464 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.134539 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.134712 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.134788 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.134899 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.135034 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.135145 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.135228 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] standard_threads = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.135380 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.135464 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.135614 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] standard_threads = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.135696 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.135791 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.135867 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.135972 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] use_journal = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.136078 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.136155 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] use_json = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.136254 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.136336 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] use_stderr = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.136433 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] use_journal = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.136518 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] use_syslog = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.136595 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] use_json = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.136698 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.136780 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] use_stderr = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.136932 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] use_syslog = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.137007 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.137176 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 16:27:15.137256 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.137443 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.137527 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.137684 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.137760 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.137935 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.138011 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.138176 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.138251 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.138426 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.138500 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.138648 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.138726 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.138900 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.138982 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.139137 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.139213 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.139387 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.139464 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.139629 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.139704 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.139874 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.139950 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.140108 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.140185 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.140356 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.140441 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.140589 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.140664 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.140837 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.140915 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.141086 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.141190 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.141351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.141438 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.141609 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.141686 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.141849 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.141932 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.142094 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.142175 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.142336 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.142413 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.142630 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.142714 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.142846 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.142922 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.143083 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.143160 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.143320 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.143396 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.143555 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.143630 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.143786 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.143873 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.144018 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.144096 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.144254 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.144408 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.144486 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.144557 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.144710 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.144788 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.144954 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.145042 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.145742 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.145821 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.145992 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.146070 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.146243 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.146331 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.146476 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.146552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.146718 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.146794 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.146947 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.147030 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.147426 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.147509 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.147667 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.147747 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.147898 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.147982 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.148128 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.148210 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.148356 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.148436 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.148588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.148673 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.148824 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.148901 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.149064 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.149205 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.149317 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.149421 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.149549 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.149656 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.149779 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.149880 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.150017 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.150127 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.150252 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.150401 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.150486 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.150606 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.150718 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.150803 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.150955 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.151033 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.151191 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.151308 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.151429 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.151554 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.151660 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.151738 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.151894 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.151977 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.152124 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.152209 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.152376 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.152494 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.152613 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.152699 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.152855 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.152932 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.153089 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.153231 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.153358 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.153439 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.153583 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.153662 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.153818 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.153901 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.154056 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.154133 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.154295 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.154380 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.154566 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.154650 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.154805 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.154880 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.155044 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.155119 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.155283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.155362 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.155517 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.155593 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.155753 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.155829 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.155988 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.156066 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.156225 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.156301 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.156462 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.156537 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.156688 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.156761 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.156929 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.157013 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.157186 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.157265 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.157418 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.157496 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.157653 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.157731 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.157892 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.157969 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.158129 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.158214 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.158377 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.158454 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.158621 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.158697 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.158862 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.158940 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.159097 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.159173 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.159334 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.159416 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.159569 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.159657 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.159802 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.159885 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.160037 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.160113 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.160270 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.160347 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.160501 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.160579 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.160805 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.161404 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.161575 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.161664 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.161810 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.161889 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.162055 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.162133 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.162292 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.162369 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.162543 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.162674 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.162788 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.162945 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.163035 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.163192 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.163418 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.163496 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] auth.external = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.163653 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.163729 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.163890 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] auth.external = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.163977 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.164125 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.164229 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.164407 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] auth.password = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.164485 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.164644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] auth.token = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.164722 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.164876 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] auth.password = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.164963 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.165111 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] auth.token = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.165240 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.165378 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.165457 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.165614 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.165702 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.165860 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.165943 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.166096 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.166167 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.166334 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.166419 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.166577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] credential.caching = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.166656 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.166808 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.166884 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.167046 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] credential.caching = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.167132 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.167296 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.167374 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.167535 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.167612 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.167782 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.167869 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.168018 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.168095 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.168259 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.168336 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.168496 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.168582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.168760 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.168837 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.169013 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.169089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.169275 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.169368 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.169533 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.169611 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.169818 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.169895 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.170061 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.170145 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.170322 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.170406 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.170559 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] federation.caching = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.170636 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.170803 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.170893 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.171038 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] federation.caching = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.171115 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.171279 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.171358 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.171513 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.171667 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.171763 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.171840 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.171995 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.172074 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.172245 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.172332 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.172483 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.172562 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.172743 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.172820 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.172985 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.173078 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.173259 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.173340 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.173500 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.173577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] identity.caching = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.173749 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.173838 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.173983 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] identity.caching = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.174060 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.174228 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.174306 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.174469 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.174555 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.174701 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.174779 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.174938 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.175021 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.175175 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.175331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.175411 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.175483 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.175633 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.175711 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.175884 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.175963 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.176116 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.176193 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.176381 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.176459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.176614 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.176691 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.176847 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.176924 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.177093 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.177207 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.177368 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.177448 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.177618 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.177695 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.177872 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.177956 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.178126 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.178254 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.178365 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.178441 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.178601 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.178680 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.178843 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.178922 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.179071 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.179147 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.179306 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.179382 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.179535 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.179619 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.179770 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.179854 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.180011 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.180089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.180249 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.180328 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.180494 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.180578 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.180726 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.180804 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.180967 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.181046 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.181232 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.181312 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.181470 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.181559 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.181702 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.181782 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.181947 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.182027 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.182175 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.182251 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.182413 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.182491 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.182643 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.182719 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.182882 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.182961 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.183120 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.183199 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.183357 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.183434 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.183596 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.183673 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.183836 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.183915 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.184083 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.184159 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.184320 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.184403 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.184550 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.184628 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.184783 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.184861 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.185033 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.185119 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.185296 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.185375 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.185530 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.185606 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.185762 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.user = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.185839 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.185995 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.186072 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.186225 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.user = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.186302 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.186465 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.186543 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.186716 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.186794 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.186945 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.187022 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.187193 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.187280 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.187430 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.187507 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.187665 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.187743 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.187896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.187982 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.188126 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.188204 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.188359 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.188443 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.188589 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.188675 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.188822 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.188900 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.189051 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.189131 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.189317 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.189412 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.189565 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.189645 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.189811 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.189888 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.190043 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.190127 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.190289 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.190366 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.190519 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.190597 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.190757 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.190842 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.190992 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.191070 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.191229 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.191307 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.191467 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.191553 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.191702 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.191781 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.191963 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.192040 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.192212 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.192297 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.192446 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.192523 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.192694 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.192771 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.192924 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.193009 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.193217 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.193300 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.193416 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.193492 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.193651 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.193741 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.193890 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.193968 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.194124 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.194201 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.194367 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.194520 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.194595 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.194667 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] resource.caching = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.194825 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.194904 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.195065 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] resource.caching = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.195152 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.195308 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.195400 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.195559 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.195688 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.195795 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.195884 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.196043 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.196122 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.196292 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.196368 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.196524 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.196610 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.196766 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.196843 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.197001 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.197079 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] role.caching = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.197265 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.197358 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] role.driver = sql {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.197502 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] role.caching = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.197581 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.197751 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] role.driver = sql {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.197829 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.197984 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.198071 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.198235 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.198313 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.198484 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.198562 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.198730 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.198816 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.198979 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.199057 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.199223 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.199301 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.199471 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.199551 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.199704 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.199782 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.199951 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.200028 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.200183 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.200262 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.200429 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.200508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.200677 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.200754 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.200920 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.201112 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.201221 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.201377 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.201603 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.201682 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.201846 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.201926 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.202099 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.202178 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.202349 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.202426 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.202588 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.202668 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.202842 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.202920 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.203078 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.203157 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.203323 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.203402 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.203561 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.203684 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.203798 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.203877 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.204028 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.204106 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.204261 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.204340 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.204493 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.204579 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.204747 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.204826 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.204990 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.205066 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.205253 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.205347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.205492 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.205577 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.205726 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.205804 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] token.caching = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.205958 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.206044 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.206193 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] token.caching = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.206272 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.206431 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.206508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.206678 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.206764 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.206915 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.206995 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.207154 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.207232 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.207404 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.207490 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.208171 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.208253 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.208420 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.208499 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.208659 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.208748 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.208903 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.208981 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.209138 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.209260 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.209398 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.209488 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.209634 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.209713 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.209866 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.209942 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.210110 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.210197 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.210337 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.210415 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.210579 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.210677 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] database.connection = **** {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.210875 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.210969 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.211139 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] database.connection = **** {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.211231 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.211386 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.211466 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.211635 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.211713 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.211870 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.211947 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.212107 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.212186 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.212342 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.212421 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.212580 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.212658 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.212814 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.212893 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.213050 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.213130 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.213321 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.213406 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.213557 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.213637 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.213803 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.213882 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.214037 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.214115 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.214282 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.214363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.214517 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.214596 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.214758 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.214835 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.214989 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.215069 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.215162 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 16:27:15.215240 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.215449 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 16:27:15.215620 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 16:27:15.247198 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.247390 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.247459 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.247600 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.247684 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.247825 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.257135 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.257291 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.286697 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.286868 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.286965 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.287045 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.341835 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.342239 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.393310 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.393490 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.394319 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.394562 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.460612 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.460798 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.460892 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.461009 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.461130 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.461264 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.461343 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.461420 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.461511 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.461591 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.461699 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.461828 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.485065 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.485312 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.485405 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.485547 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.485635 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.485771 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.485855 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.485992 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.486076 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.486210 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.486292 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.486429 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.486519 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.486680 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.497916 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.498216 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.513028 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.513273 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.513482 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.513575 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.513705 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.513793 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.513928 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.514013 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.514146 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.514230 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.514359 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.514442 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.514607 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.514788 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 16:27:15.532442 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.532631 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.532763 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 16:27:15.532909 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 16:27:15.533051 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.533227 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.533359 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 16:27:15.533492 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 16:27:15.533625 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.533765 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.533896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.534026 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.534154 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.534281 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.534417 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.534548 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.545216 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.545352 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.545480 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.545613 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.545757 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.545890 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.546023 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.546156 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.554131 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 16:27:15.554285 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 16:27:15.557423 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 16:27:15.557621 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 16:27:15.557752 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.557885 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.558029 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.558161 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.558347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.558555 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.558684 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.558848 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.558983 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.559113 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.559255 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.559388 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.559521 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.559659 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.559791 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.559921 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.560053 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.560186 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.560321 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.560452 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.587968 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.588224 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.589817 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 16:27:15.589987 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 16:27:15.590235 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.590391 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.595565 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 16:27:15.595719 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 16:27:15.595851 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.595999 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.596134 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.596283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.603758 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 16:27:15.604057 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 16:27:15.604246 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.604404 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.609242 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 16:27:15.609490 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 16:27:15.609628 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.609787 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.609921 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.610046 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.610201 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.610331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.610460 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.610594 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.610720 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.610851 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.610980 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.611109 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.611245 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.611376 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.630924 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.631703 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.631845 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.631998 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.635718 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 16:27:15.635971 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 16:27:15.639325 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.639500 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.639654 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.639791 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.644594 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 16:27:15.644799 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 16:27:15.648535 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 16:27:15.648774 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 16:27:15.652389 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.652583 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.652716 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.652885 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.653017 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.653251 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.653396 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.653530 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.653665 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.653864 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.654007 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.654142 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.672702 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 16:27:15.673029 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 16:27:15.673186 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.673360 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.673500 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.673630 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.673760 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.673956 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.682030 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.682411 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.683752 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.684030 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.684162 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.684402 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.687101 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 16:27:15.687389 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.687531 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 16:27:15.687667 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.687797 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.687944 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.688078 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.688214 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.688345 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.688574 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.702114 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 16:27:15.702390 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.702550 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 16:27:15.702674 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.702815 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.702954 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.703084 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.703213 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.703343 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.703477 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.703620 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.703793 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.703922 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.704082 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.721783 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 16:27:15.722131 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 16:27:15.725058 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 16:27:15.725441 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 16:27:15.728366 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.728724 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.729985 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.730362 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.731635 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 16:27:15.731917 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.732007 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 16:27:15.732201 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.732312 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.732547 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.740123 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.740490 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.741800 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 16:27:15.742281 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 16:27:15.744990 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.745300 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.745589 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.745872 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.746009 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.746241 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.746524 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.746800 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.755598 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 16:27:15.755889 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.756227 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.756543 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 16:27:15.756884 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.757241 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.764227 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 16:27:15.764516 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.764797 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.765073 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.765222 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 16:27:15.765376 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.765504 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.765667 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.765797 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.765950 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.766089 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.766217 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.766356 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.766483 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.766611 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.766751 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.766958 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.767091 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.767172 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.767247 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.767322 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.767490 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.767574 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.767748 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.767829 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.768045 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.768319 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.768582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 16:27:15.807152 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5643111049c0 pid: 80466 (default app) May 19 16:27:15.807911 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5643111049c0 pid: 80467 (default app) May 19 16:27:15.810598 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55b98e9-b98f-4017-ad08-4004e36c497d None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:15.810895 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55b98e9-b98f-4017-ad08-4004e36c497d None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:15.811178 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55b98e9-b98f-4017-ad08-4004e36c497d None None] PATH_INFO: `/v3` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:15.815438 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1/1] 10.222.0.36 () {50 vars in 791 bytes} [Thu May 19 16:27:15 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:23.942343 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db1320-a00f-43ee-bd93-fe772e0ef323 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:23.942892 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db1320-a00f-43ee-bd93-fe772e0ef323 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:23.942892 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db1320-a00f-43ee-bd93-fe772e0ef323 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:23.948185 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1/2] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:27:23 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 16:27:23.953887 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a3685-e7b6-42cb-a0d1-f99009281c7b None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:23.954334 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a3685-e7b6-42cb-a0d1-f99009281c7b None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:23.954513 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a3685-e7b6-42cb-a0d1-f99009281c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:24.023698 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: INFO dbcounter [None req-ac2a3685-e7b6-42cb-a0d1-f99009281c7b None None] Registered counter for database keystone May 19 16:27:24.045833 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG oslo_db.sqlalchemy.engines [None req-ac2a3685-e7b6-42cb-a0d1-f99009281c7b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80466) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 16:27:24.052139 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writer thread running {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 16:27:24.237934 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG passlib.handlers.bcrypt [None req-ac2a3685-e7b6-42cb-a0d1-f99009281c7b None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80466) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 16:27:24.238272 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG passlib.handlers.bcrypt [None req-ac2a3685-e7b6-42cb-a0d1-f99009281c7b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80466) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 16:27:24.294608 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-ac2a3685-e7b6-42cb-a0d1-f99009281c7b None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:27:24.296222 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-ac2a3685-e7b6-42cb-a0d1-f99009281c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:27:24.421510 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 2/3] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:27:23 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 469 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 19 16:27:24.606919 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2db1320-a00f-43ee-bd93-fe772e0ef323 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:24.678059 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: INFO dbcounter [None req-a2db1320-a00f-43ee-bd93-fe772e0ef323 None None] Registered counter for database keystone May 19 16:27:24.698526 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG oslo_db.sqlalchemy.engines [None req-a2db1320-a00f-43ee-bd93-fe772e0ef323 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80467) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 16:27:24.704837 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writer thread running {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 16:27:24.829482 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2db1320-a00f-43ee-bd93-fe772e0ef323 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:24.833749 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe54634f-6e3c-4eda-a9fc-e4ef2e6f864f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:24.834417 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54634f-6e3c-4eda-a9fc-e4ef2e6f864f admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:24.834705 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54634f-6e3c-4eda-a9fc-e4ef2e6f864f admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:24.834987 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54634f-6e3c-4eda-a9fc-e4ef2e6f864f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:24.843732 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe54634f-6e3c-4eda-a9fc-e4ef2e6f864f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:24.894844 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: WARNING keystone.common.rbac_enforcer.enforcer [None req-fe54634f-6e3c-4eda-a9fc-e4ef2e6f864f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 16:27:24.895190 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe54634f-6e3c-4eda-a9fc-e4ef2e6f864f admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:24.905943 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 2/4] 10.222.0.36 () {58 vars in 1324 bytes} [Thu May 19 16:27:24 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 16:27:24.911947 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2a3685-e7b6-42cb-a0d1-f99009281c7b None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:24.927119 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2a3685-e7b6-42cb-a0d1-f99009281c7b None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:24.931550 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f694b389-84a8-4c7d-afe1-1447bb26b85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:24.932674 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694b389-84a8-4c7d-afe1-1447bb26b85a admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:24.932959 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694b389-84a8-4c7d-afe1-1447bb26b85a admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:24.933262 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694b389-84a8-4c7d-afe1-1447bb26b85a admin admin] PATH_INFO: `/v3/projects/8d5de9b2c50341a8b78fb369c7765787` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:24.935175 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f694b389-84a8-4c7d-afe1-1447bb26b85a admin admin] RBAC: Authorizing `identity:get_project(project_id=8d5de9b2c50341a8b78fb369c7765787)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:25.093960 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f694b389-84a8-4c7d-afe1-1447bb26b85a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 16:27:25.094631 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f694b389-84a8-4c7d-afe1-1447bb26b85a admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:25.099561 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 3/5] 10.222.0.36 () {56 vars in 1207 bytes} [Thu May 19 16:27:24 2022] GET /identity/v3/projects/8d5de9b2c50341a8b78fb369c7765787 => generated 344 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:27.295462 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a4bd7c-d6c6-4a10-9bb9-148423c3bec0 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:27.295952 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a4bd7c-d6c6-4a10-9bb9-148423c3bec0 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:27.296060 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a4bd7c-d6c6-4a10-9bb9-148423c3bec0 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:27.300192 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 3/6] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:27:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 16:27:27.305802 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db98bbd-b4dd-424a-aa38-b9192823f54b None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:27.306206 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db98bbd-b4dd-424a-aa38-b9192823f54b None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:27.306358 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db98bbd-b4dd-424a-aa38-b9192823f54b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:27.325436 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-8db98bbd-b4dd-424a-aa38-b9192823f54b None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:27:27.326970 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-8db98bbd-b4dd-424a-aa38-b9192823f54b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:27:27.335401 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 4/7] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:27:27 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 19 16:27:27.357173 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a4bd7c-d6c6-4a10-9bb9-148423c3bec0 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:27.369531 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a4bd7c-d6c6-4a10-9bb9-148423c3bec0 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:27.373828 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38f2588-4a65-4aa2-9477-af25a7133034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:27.374475 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f2588-4a65-4aa2-9477-af25a7133034 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:27.374763 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f2588-4a65-4aa2-9477-af25a7133034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:27.375043 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f2588-4a65-4aa2-9477-af25a7133034 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:27.383499 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38f2588-4a65-4aa2-9477-af25a7133034 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:27.385315 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38f2588-4a65-4aa2-9477-af25a7133034 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:27.394234 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 4/8] 10.222.0.36 () {58 vars in 1324 bytes} [Thu May 19 16:27:27 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 16:27:27.401066 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db98bbd-b4dd-424a-aa38-b9192823f54b None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:27.410264 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db98bbd-b4dd-424a-aa38-b9192823f54b None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:27.415019 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518693a8-3d54-4e8e-afae-d1e6b1121b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:27.415895 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518693a8-3d54-4e8e-afae-d1e6b1121b7e admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:27.416184 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518693a8-3d54-4e8e-afae-d1e6b1121b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:27.416466 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518693a8-3d54-4e8e-afae-d1e6b1121b7e admin admin] PATH_INFO: `/v3/users/ac4bfc4d6198473c8d280a14b2c6036f` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:27.417962 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518693a8-3d54-4e8e-afae-d1e6b1121b7e admin admin] RBAC: Authorizing `identity:get_user(user_id=ac4bfc4d6198473c8d280a14b2c6036f)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:27.419827 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518693a8-3d54-4e8e-afae-d1e6b1121b7e admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:27.421469 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 5/9] 10.222.0.36 () {56 vars in 1198 bytes} [Thu May 19 16:27:27 2022] GET /identity/v3/users/ac4bfc4d6198473c8d280a14b2c6036f => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:29.718132 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f1639d-6ab8-41b9-b810-fef65d1255f1 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:29.718754 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f1639d-6ab8-41b9-b810-fef65d1255f1 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:29.718754 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f1639d-6ab8-41b9-b810-fef65d1255f1 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:29.720054 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 5/10] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:27:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:27:29.725609 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac98b8-8bb0-4908-8fe9-02382ae6b193 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:29.726074 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac98b8-8bb0-4908-8fe9-02382ae6b193 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:29.726169 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac98b8-8bb0-4908-8fe9-02382ae6b193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:29.739389 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4b8f6-eee0-4d1a-b221-1cd79c22e9e8 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:29.739675 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4b8f6-eee0-4d1a-b221-1cd79c22e9e8 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:29.739950 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4b8f6-eee0-4d1a-b221-1cd79c22e9e8 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:29.741332 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 6/11] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:27:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 16:27:29.744151 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-a8ac98b8-8bb0-4908-8fe9-02382ae6b193 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:27:29.745443 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-a8ac98b8-8bb0-4908-8fe9-02382ae6b193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:27:29.748822 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f99a2-a751-4d6f-bd6a-3abc0e62b460 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:29.749309 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f99a2-a751-4d6f-bd6a-3abc0e62b460 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:29.749727 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f99a2-a751-4d6f-bd6a-3abc0e62b460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:29.799774 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG passlib.handlers.bcrypt [None req-5b1f99a2-a751-4d6f-bd6a-3abc0e62b460 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80467) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 16:27:29.800216 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG passlib.handlers.bcrypt [None req-5b1f99a2-a751-4d6f-bd6a-3abc0e62b460 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80467) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 16:27:29.829907 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 6/12] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:27:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:27:29.847310 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-5b1f99a2-a751-4d6f-bd6a-3abc0e62b460 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:27:29.848944 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-5b1f99a2-a751-4d6f-bd6a-3abc0e62b460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:27:29.854614 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac98b8-8bb0-4908-8fe9-02382ae6b193 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:29.880371 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:29.955266 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 7/13] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:27:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 16:27:29.964739 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac98b8-8bb0-4908-8fe9-02382ae6b193 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:29.977606 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f99a2-a751-4d6f-bd6a-3abc0e62b460 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:30.004070 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c89c5-7cda-43d0-8ba5-44b32d23d14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:30.004907 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c89c5-7cda-43d0-8ba5-44b32d23d14a None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:30.005229 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c89c5-7cda-43d0-8ba5-44b32d23d14a None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:30.005514 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c89c5-7cda-43d0-8ba5-44b32d23d14a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:30.017275 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3c89c5-7cda-43d0-8ba5-44b32d23d14a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:27:30.028821 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-5e3c89c5-7cda-43d0-8ba5-44b32d23d14a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:27:30.028821 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:27:30.028821 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:27:30.028821 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:27:30.028821 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:27:30.028821 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:27:30.028821 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:27:30.028821 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:27:30.028821 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:27:30.028821 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:27:30.028821 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:27:30.028821 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:27:30.028821 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:27:30.028821 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:27:30.028821 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:27:30.028821 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:27:30.029865 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:27:30.029865 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:27:30.029865 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:27:30.029865 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:27:30.029865 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:27:30.029865 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:27:30.029865 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:27:30.029865 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:27:30.029865 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:27:30.029865 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:27:30.029865 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:27:30.029865 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:27:30.029865 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:27:30.029865 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:27:30.029865 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:27:30.029865 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:27:30.030707 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:27:30.030707 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:27:30.030707 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:27:30.030707 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:27:30.030707 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:27:30.030707 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:27:30.030707 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:27:30.030707 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:27:30.030707 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:27:30.030707 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:27:30.030707 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:27:30.030707 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:27:30.030707 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 7/14] 10.222.0.36 () {56 vars in 1096 bytes} [Thu May 19 16:27:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:27:30.035965 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c89c5-7cda-43d0-8ba5-44b32d23d14a None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:30.130395 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:30.156157 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c89c5-7cda-43d0-8ba5-44b32d23d14a None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:30.196677 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f740a5cc-da09-405b-8e01-66ef01b3b4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:30.197384 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740a5cc-da09-405b-8e01-66ef01b3b4b9 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:30.197672 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740a5cc-da09-405b-8e01-66ef01b3b4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:30.197951 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740a5cc-da09-405b-8e01-66ef01b3b4b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:30.198715 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f740a5cc-da09-405b-8e01-66ef01b3b4b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:27:30.199021 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f740a5cc-da09-405b-8e01-66ef01b3b4b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:30.200456 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f740a5cc-da09-405b-8e01-66ef01b3b4b9 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:30.213115 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 8/15] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:27:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:27:30.218502 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f740a5cc-da09-405b-8e01-66ef01b3b4b9 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:30.227378 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f99a2-a751-4d6f-bd6a-3abc0e62b460 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:30.234519 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:30.275462 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea33f9-a912-49d0-890c-c021da4f52e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:30.276363 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea33f9-a912-49d0-890c-c021da4f52e1 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:30.276650 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea33f9-a912-49d0-890c-c021da4f52e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:30.276927 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea33f9-a912-49d0-890c-c021da4f52e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:30.337164 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f740a5cc-da09-405b-8e01-66ef01b3b4b9 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:30.346168 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fea33f9-a912-49d0-890c-c021da4f52e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:30.348227 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fea33f9-a912-49d0-890c-c021da4f52e1 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:30.379211 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d46ed7-067c-4973-91d6-29aa797d26ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:30.380448 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d46ed7-067c-4973-91d6-29aa797d26ea None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:30.380860 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d46ed7-067c-4973-91d6-29aa797d26ea None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:30.381284 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d46ed7-067c-4973-91d6-29aa797d26ea None admin] PATH_INFO: `/v3/users/ac4bfc4d6198473c8d280a14b2c6036f` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:30.382970 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d46ed7-067c-4973-91d6-29aa797d26ea None admin] RBAC: Authorizing `identity:get_user(user_id=ac4bfc4d6198473c8d280a14b2c6036f)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:30.385091 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d46ed7-067c-4973-91d6-29aa797d26ea None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:30.387538 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 9/16] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:27:30 2022] GET /identity/v3/users/ac4bfc4d6198473c8d280a14b2c6036f => generated 247 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:27:30.393094 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d46ed7-067c-4973-91d6-29aa797d26ea None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:30.417295 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:30.471346 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 8/17] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:27:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 16:27:30.477734 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea33f9-a912-49d0-890c-c021da4f52e1 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:30.480699 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:30.506212 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d46ed7-067c-4973-91d6-29aa797d26ea None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:30.553068 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f84c2e-13f1-408e-b22d-794d1daf0870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:30.553910 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f84c2e-13f1-408e-b22d-794d1daf0870 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:30.554198 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f84c2e-13f1-408e-b22d-794d1daf0870 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:30.554478 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f84c2e-13f1-408e-b22d-794d1daf0870 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:30.555713 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f84c2e-13f1-408e-b22d-794d1daf0870 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:30.557239 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f84c2e-13f1-408e-b22d-794d1daf0870 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:30.559074 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 10/18] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:27:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:27:30.564576 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f84c2e-13f1-408e-b22d-794d1daf0870 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:30.588523 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea33f9-a912-49d0-890c-c021da4f52e1 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:30.625066 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97124361-73a5-4882-845d-a839d9f148bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:30.626200 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97124361-73a5-4882-845d-a839d9f148bc None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:30.626624 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97124361-73a5-4882-845d-a839d9f148bc None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:30.627086 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97124361-73a5-4882-845d-a839d9f148bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:30.646301 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:30.656445 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97124361-73a5-4882-845d-a839d9f148bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:30.658999 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97124361-73a5-4882-845d-a839d9f148bc None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:30.668259 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f84c2e-13f1-408e-b22d-794d1daf0870 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:30.688920 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-97124361-73a5-4882-845d-a839d9f148bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:27:30.688920 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:27:30.688920 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:27:30.688920 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:27:30.688920 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:27:30.688920 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:27:30.688920 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:27:30.688920 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:27:30.688920 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:27:30.688920 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:27:30.688920 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:27:30.688920 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:27:30.688920 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:27:30.688920 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:27:30.688920 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:27:30.689957 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:27:30.689957 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:27:30.689957 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:27:30.689957 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:27:30.689957 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:27:30.689957 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:27:30.689957 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:27:30.689957 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:27:30.689957 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:27:30.689957 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:27:30.689957 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:27:30.689957 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:27:30.689957 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:27:30.689957 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:27:30.689957 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:27:30.689957 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:27:30.689957 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:27:30.690741 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:27:30.690741 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:27:30.690741 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:27:30.690741 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:27:30.691256 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 9/19] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:27:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 16:27:30.696617 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97124361-73a5-4882-845d-a839d9f148bc None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:30.701911 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4007784d-45db-4a09-bfa4-5d5d87632e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:30.702579 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007784d-45db-4a09-bfa4-5d5d87632e0b None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:30.702865 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007784d-45db-4a09-bfa4-5d5d87632e0b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:30.703141 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007784d-45db-4a09-bfa4-5d5d87632e0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:30.703985 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4007784d-45db-4a09-bfa4-5d5d87632e0b None admin] RBAC: Adding query filter params (role.id=e2915fe109244a07a9be96bd2b219fdd, scope.domain.id=default, user.id=ac4bfc4d6198473c8d280a14b2c6036f) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:27:30.704295 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4007784d-45db-4a09-bfa4-5d5d87632e0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:30.705820 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4007784d-45db-4a09-bfa4-5d5d87632e0b None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:30.714826 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 11/20] 10.222.0.36 () {56 vars in 1322 bytes} [Thu May 19 16:27:30 2022] GET /identity/v3/role_assignments?user.id=ac4bfc4d6198473c8d280a14b2c6036f&scope.domain.id=default&role.id=e2915fe109244a07a9be96bd2b219fdd => generated 233 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:27:30.772853 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:30.823020 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97124361-73a5-4882-845d-a839d9f148bc None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:30.864078 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f1d0b1-3aac-4a43-966d-832d44374d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:30.864756 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1d0b1-3aac-4a43-966d-832d44374d51 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:30.865045 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1d0b1-3aac-4a43-966d-832d44374d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:30.865355 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1d0b1-3aac-4a43-966d-832d44374d51 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:30.866124 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f1d0b1-3aac-4a43-966d-832d44374d51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:27:30.866437 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f1d0b1-3aac-4a43-966d-832d44374d51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:30.867883 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f1d0b1-3aac-4a43-966d-832d44374d51 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:30.889619 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 10/21] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:27:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:33.525544 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506d611-4259-425e-901f-8ecb9c17fa63 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:33.526571 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506d611-4259-425e-901f-8ecb9c17fa63 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:33.527059 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506d611-4259-425e-901f-8ecb9c17fa63 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:33.529022 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 12/22] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:27:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:27:33.534276 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435e9cc-b517-47b0-8570-25d7567469d3 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:33.534763 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435e9cc-b517-47b0-8570-25d7567469d3 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:33.534843 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435e9cc-b517-47b0-8570-25d7567469d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:33.570224 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-2435e9cc-b517-47b0-8570-25d7567469d3 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:27:33.571472 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-2435e9cc-b517-47b0-8570-25d7567469d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:27:33.662549 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 11/23] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:27:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:27:33.720753 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6506d611-4259-425e-901f-8ecb9c17fa63 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:33.867596 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6506d611-4259-425e-901f-8ecb9c17fa63 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:33.926671 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632732d8-8095-4a68-8874-71058ee18279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:33.929328 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632732d8-8095-4a68-8874-71058ee18279 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:33.929627 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632732d8-8095-4a68-8874-71058ee18279 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:33.929881 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632732d8-8095-4a68-8874-71058ee18279 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:33.947280 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632732d8-8095-4a68-8874-71058ee18279 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:27:33.955084 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-632732d8-8095-4a68-8874-71058ee18279 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:27:33.955084 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:27:33.955084 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:27:33.955084 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:27:33.955084 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:27:33.955084 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:27:33.955084 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:27:33.955084 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:27:33.955084 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:27:33.955084 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:27:33.955084 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:27:33.955084 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:27:33.955084 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:27:33.955084 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:27:33.955084 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:27:33.955084 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:27:33.956480 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:27:33.956480 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:27:33.956480 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:27:33.956480 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:27:33.956480 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:27:33.956480 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:27:33.956480 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:27:33.956480 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:27:33.956480 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:27:33.956480 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:27:33.956480 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:27:33.956480 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:27:33.956480 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:27:33.956480 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:27:33.956480 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:27:33.956480 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:27:33.957719 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:27:33.957719 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:27:33.957719 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:27:33.957719 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:27:33.957719 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:27:33.957719 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:27:33.957719 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:27:33.957719 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:27:33.957719 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:27:33.957719 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:27:33.957719 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:27:33.957719 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:27:33.959836 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 13/24] 10.222.0.36 () {56 vars in 1096 bytes} [Thu May 19 16:27:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:27:33.966429 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2435e9cc-b517-47b0-8570-25d7567469d3 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:34.008690 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a227842-e6e0-4e7d-a533-4611f61af0bc None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:34.009176 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a227842-e6e0-4e7d-a533-4611f61af0bc None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:34.009916 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a227842-e6e0-4e7d-a533-4611f61af0bc None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:34.012243 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 14/25] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:27:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 16:27:34.019796 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be087189-af28-46f2-b411-b3a707f0e552 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:34.020637 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be087189-af28-46f2-b411-b3a707f0e552 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:34.021214 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be087189-af28-46f2-b411-b3a707f0e552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:34.045935 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:34.058193 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-be087189-af28-46f2-b411-b3a707f0e552 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:27:34.059438 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-be087189-af28-46f2-b411-b3a707f0e552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:27:34.069637 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:34.180760 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2435e9cc-b517-47b0-8570-25d7567469d3 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:34.191940 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 15/26] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:27:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 16:27:34.193558 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652f5f8-ba21-4b27-a1e8-86c995d5aa7c None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:34.193853 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652f5f8-ba21-4b27-a1e8-86c995d5aa7c None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:34.194132 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652f5f8-ba21-4b27-a1e8-86c995d5aa7c None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:34.195413 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 16/27] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:27:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:27:34.201429 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b771938f-fe0d-4112-8341-41f6d9a809c4 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:34.201953 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b771938f-fe0d-4112-8341-41f6d9a809c4 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:34.202470 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b771938f-fe0d-4112-8341-41f6d9a809c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:34.230486 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-b771938f-fe0d-4112-8341-41f6d9a809c4 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:27:34.232036 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b771938f-fe0d-4112-8341-41f6d9a809c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:27:34.232707 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6acf79e-d391-4345-af7c-d42870f9acd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:34.233717 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acf79e-d391-4345-af7c-d42870f9acd0 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:34.234163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acf79e-d391-4345-af7c-d42870f9acd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:34.234695 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acf79e-d391-4345-af7c-d42870f9acd0 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:34.235700 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6acf79e-d391-4345-af7c-d42870f9acd0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:27:34.236202 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6acf79e-d391-4345-af7c-d42870f9acd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:34.239085 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6acf79e-d391-4345-af7c-d42870f9acd0 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:34.266614 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 12/28] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:27:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:34.267942 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6acf79e-d391-4345-af7c-d42870f9acd0 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:34.349789 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 17/29] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:27:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:27:34.357188 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b771938f-fe0d-4112-8341-41f6d9a809c4 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:34.443433 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6acf79e-d391-4345-af7c-d42870f9acd0 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:34.458319 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:34.458504 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:34.506351 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab2248-942b-4684-aa8d-ad7b048cc9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:34.507895 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab2248-942b-4684-aa8d-ad7b048cc9de None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:34.508434 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab2248-942b-4684-aa8d-ad7b048cc9de None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:34.508762 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab2248-942b-4684-aa8d-ad7b048cc9de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:34.513727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b771938f-fe0d-4112-8341-41f6d9a809c4 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:34.523523 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ab2248-942b-4684-aa8d-ad7b048cc9de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:34.525635 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ab2248-942b-4684-aa8d-ad7b048cc9de None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:34.554252 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-66ab2248-942b-4684-aa8d-ad7b048cc9de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:27:34.554252 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:27:34.554252 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:27:34.554252 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:27:34.554252 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:27:34.554252 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:27:34.554252 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:27:34.554252 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:27:34.554252 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:27:34.554252 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:27:34.554252 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:27:34.554252 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:27:34.554252 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:27:34.554252 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:27:34.554252 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:27:34.555648 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:27:34.555648 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:27:34.555648 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:27:34.555648 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:27:34.555648 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:27:34.555648 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:27:34.555648 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:27:34.555648 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:27:34.555648 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:27:34.555648 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:27:34.555648 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:27:34.555648 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:27:34.555648 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:27:34.555648 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:27:34.555648 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:27:34.555648 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:27:34.555648 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:27:34.556705 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:27:34.556705 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:27:34.556705 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:27:34.556705 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:27:34.559117 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 13/30] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:27:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:27:34.559306 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf21d7-bf21-46f1-ad82-7624ff1321de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:34.560597 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf21d7-bf21-46f1-ad82-7624ff1321de None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:34.561008 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf21d7-bf21-46f1-ad82-7624ff1321de None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:34.561414 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf21d7-bf21-46f1-ad82-7624ff1321de None admin] PATH_INFO: `/v3/users/ac4bfc4d6198473c8d280a14b2c6036f` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:34.561986 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9227d06-d9b8-4959-ab44-d8c5c910d349 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:34.562453 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9227d06-d9b8-4959-ab44-d8c5c910d349 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:34.562969 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cf21d7-bf21-46f1-ad82-7624ff1321de None admin] RBAC: Authorizing `identity:get_user(user_id=ac4bfc4d6198473c8d280a14b2c6036f)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:34.563097 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9227d06-d9b8-4959-ab44-d8c5c910d349 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:34.564838 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 14/31] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:27:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:27:34.566032 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9227d06-d9b8-4959-ab44-d8c5c910d349 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:34.570023 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cf21d7-bf21-46f1-ad82-7624ff1321de None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:34.573134 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 18/32] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:27:34 2022] GET /identity/v3/users/ac4bfc4d6198473c8d280a14b2c6036f => generated 247 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:34.574316 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf21d7-bf21-46f1-ad82-7624ff1321de None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:34.708771 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9227d06-d9b8-4959-ab44-d8c5c910d349 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:34.724327 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf21d7-bf21-46f1-ad82-7624ff1321de None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:34.752658 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0532f5aa-fe5c-468a-945b-3a01f07501e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:34.754377 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532f5aa-fe5c-468a-945b-3a01f07501e9 None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:34.754901 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532f5aa-fe5c-468a-945b-3a01f07501e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:34.755393 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532f5aa-fe5c-468a-945b-3a01f07501e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:34.757083 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0532f5aa-fe5c-468a-945b-3a01f07501e9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:34.759323 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0532f5aa-fe5c-468a-945b-3a01f07501e9 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:34.768388 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:34.788894 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf17c1-5554-4b27-8abc-52a557293964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:34.798073 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf17c1-5554-4b27-8abc-52a557293964 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:34.798395 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf17c1-5554-4b27-8abc-52a557293964 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:34.798641 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf17c1-5554-4b27-8abc-52a557293964 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:34.799417 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bf17c1-5554-4b27-8abc-52a557293964 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:27:34.799754 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bf17c1-5554-4b27-8abc-52a557293964 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:34.801252 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bf17c1-5554-4b27-8abc-52a557293964 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:34.820280 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 19/33] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:27:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:34.821479 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fafd227-af96-4cd3-bda6-978b9c038894 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:34.821775 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fafd227-af96-4cd3-bda6-978b9c038894 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:34.822059 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fafd227-af96-4cd3-bda6-978b9c038894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:34.841950 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-2fafd227-af96-4cd3-bda6-978b9c038894 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:27:34.843229 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-2fafd227-af96-4cd3-bda6-978b9c038894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:27:34.931217 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 20/34] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:27:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:27:34.932073 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fafd227-af96-4cd3-bda6-978b9c038894 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:34.936370 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 15/35] 10.222.0.36 () {60 vars in 1131 bytes} [Thu May 19 16:27:34 2022] POST /identity/v3/roles => generated 219 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:27:34.937824 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0532f5aa-fe5c-468a-945b-3a01f07501e9 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:34.970631 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:35.074145 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0532f5aa-fe5c-468a-945b-3a01f07501e9 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:35.117007 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fafd227-af96-4cd3-bda6-978b9c038894 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:35.117798 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2a5e16-cb4f-4031-bf2a-53464a47d8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:35.118817 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a5e16-cb4f-4031-bf2a-53464a47d8fc None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:35.119429 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a5e16-cb4f-4031-bf2a-53464a47d8fc None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:35.119906 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a5e16-cb4f-4031-bf2a-53464a47d8fc None admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:35.121181 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2a5e16-cb4f-4031-bf2a-53464a47d8fc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:35.123185 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2a5e16-cb4f-4031-bf2a-53464a47d8fc None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:35.156521 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:35.169235 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ca925b-97e3-4d60-ba63-1b77586f6da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:35.170065 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca925b-97e3-4d60-ba63-1b77586f6da5 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:35.170370 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca925b-97e3-4d60-ba63-1b77586f6da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:35.170669 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca925b-97e3-4d60-ba63-1b77586f6da5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:35.171937 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ca925b-97e3-4d60-ba63-1b77586f6da5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:35.173315 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 16/36] 10.222.0.36 () {60 vars in 1141 bytes} [Thu May 19 16:27:34 2022] POST /identity/v3/projects => generated 301 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:27:35.173930 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ca925b-97e3-4d60-ba63-1b77586f6da5 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:35.175126 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2a5e16-cb4f-4031-bf2a-53464a47d8fc None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:35.176188 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 21/37] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:27:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:35.177556 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367deeb-8b3c-46a9-bf69-e7e7fefe0890 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:35.177930 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367deeb-8b3c-46a9-bf69-e7e7fefe0890 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:35.178137 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367deeb-8b3c-46a9-bf69-e7e7fefe0890 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:35.179523 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 22/38] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:27:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:27:35.184768 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a367deeb-8b3c-46a9-bf69-e7e7fefe0890 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:35.273038 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:35.305334 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a367deeb-8b3c-46a9-bf69-e7e7fefe0890 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:35.332438 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2a5e16-cb4f-4031-bf2a-53464a47d8fc None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:35.348555 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c471bf96-cdcb-4ec6-ba9d-4c13cd1876fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:35.349524 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c471bf96-cdcb-4ec6-ba9d-4c13cd1876fb None admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:35.349815 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c471bf96-cdcb-4ec6-ba9d-4c13cd1876fb None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:35.350480 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c471bf96-cdcb-4ec6-ba9d-4c13cd1876fb None admin] PATH_INFO: `/v3/domains/default/users/ac4bfc4d6198473c8d280a14b2c6036f/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:35.352897 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c471bf96-cdcb-4ec6-ba9d-4c13cd1876fb None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=ac4bfc4d6198473c8d280a14b2c6036f, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:35.354643 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c471bf96-cdcb-4ec6-ba9d-4c13cd1876fb None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:35.390618 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7cef07-5075-4abe-addf-835868bc29e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:35.391858 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7cef07-5075-4abe-addf-835868bc29e0 None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:35.392437 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7cef07-5075-4abe-addf-835868bc29e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:35.392948 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7cef07-5075-4abe-addf-835868bc29e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:35.393983 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7cef07-5075-4abe-addf-835868bc29e0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:35.396218 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7cef07-5075-4abe-addf-835868bc29e0 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:35.413098 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 17/39] 10.222.0.36 () {60 vars in 1131 bytes} [Thu May 19 16:27:35 2022] POST /identity/v3/roles => generated 225 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:27:35.415435 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01324dca-58d8-4a40-992e-11eb24524c22 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:35.416045 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01324dca-58d8-4a40-992e-11eb24524c22 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:35.416551 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01324dca-58d8-4a40-992e-11eb24524c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:35.438294 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-01324dca-58d8-4a40-992e-11eb24524c22 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:27:35.440517 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-01324dca-58d8-4a40-992e-11eb24524c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:27:35.499845 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 23/40] 10.222.0.36 () {58 vars in 1361 bytes} [Thu May 19 16:27:35 2022] PUT /identity/v3/domains/default/users/ac4bfc4d6198473c8d280a14b2c6036f/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:27:35.529938 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 18/41] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:27:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:27:35.556642 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c471bf96-cdcb-4ec6-ba9d-4c13cd1876fb None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:35.571378 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db4fd1-ce97-406a-b1d7-976d77861ead None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:35.571913 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db4fd1-ce97-406a-b1d7-976d77861ead None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:35.572148 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db4fd1-ce97-406a-b1d7-976d77861ead None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:35.574117 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 19/42] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:27:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:27:35.582024 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c3a41-e010-4e33-8f04-7f50327211ae None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:35.582350 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c3a41-e010-4e33-8f04-7f50327211ae None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:35.582964 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c3a41-e010-4e33-8f04-7f50327211ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:35.609550 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-ab8c3a41-e010-4e33-8f04-7f50327211ae None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:27:35.611081 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ab8c3a41-e010-4e33-8f04-7f50327211ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:27:35.625620 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:35.687044 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c471bf96-cdcb-4ec6-ba9d-4c13cd1876fb None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:35.693228 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:35.709839 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 20/43] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:27:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 16:27:35.728006 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac202460-c81a-4a2d-9c3b-8f958fdb2e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:35.728718 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac202460-c81a-4a2d-9c3b-8f958fdb2e25 None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:35.729018 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac202460-c81a-4a2d-9c3b-8f958fdb2e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:35.729346 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac202460-c81a-4a2d-9c3b-8f958fdb2e25 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:35.730164 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac202460-c81a-4a2d-9c3b-8f958fdb2e25 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:35.732808 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac202460-c81a-4a2d-9c3b-8f958fdb2e25 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:35.735168 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c3a41-e010-4e33-8f04-7f50327211ae None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:35.762055 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 24/44] 10.222.0.36 () {60 vars in 1131 bytes} [Thu May 19 16:27:35 2022] POST /identity/v3/roles => generated 223 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:27:35.869442 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c3a41-e010-4e33-8f04-7f50327211ae None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:35.919918 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace46c9d-5664-4655-a9f6-9080bd6d7142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:35.921098 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace46c9d-5664-4655-a9f6-9080bd6d7142 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:35.921437 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace46c9d-5664-4655-a9f6-9080bd6d7142 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:35.921723 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace46c9d-5664-4655-a9f6-9080bd6d7142 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:35.926275 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace46c9d-5664-4655-a9f6-9080bd6d7142 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:35.927880 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace46c9d-5664-4655-a9f6-9080bd6d7142 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:35.935362 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 21/45] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:27:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:27:35.937763 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac202460-c81a-4a2d-9c3b-8f958fdb2e25 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:36.052771 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac202460-c81a-4a2d-9c3b-8f958fdb2e25 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:36.063622 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:36.090624 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46fbda3-804b-4a2f-835c-322c4cc6c148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:36.091309 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46fbda3-804b-4a2f-835c-322c4cc6c148 None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:36.091684 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46fbda3-804b-4a2f-835c-322c4cc6c148 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:36.091891 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46fbda3-804b-4a2f-835c-322c4cc6c148 None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:36.092637 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46fbda3-804b-4a2f-835c-322c4cc6c148 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:36.094172 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46fbda3-804b-4a2f-835c-322c4cc6c148 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:36.122785 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 25/46] 10.222.0.36 () {60 vars in 1141 bytes} [Thu May 19 16:27:35 2022] POST /identity/v3/projects => generated 312 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:27:37.871020 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199c581-f2b5-4d5d-ae50-586bc4b386b5 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:37.871540 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199c581-f2b5-4d5d-ae50-586bc4b386b5 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:37.871657 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199c581-f2b5-4d5d-ae50-586bc4b386b5 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:37.872971 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 22/47] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:27:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:27:37.879268 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0599e6-231f-4e07-beeb-017a8c6ecf32 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:37.879698 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0599e6-231f-4e07-beeb-017a8c6ecf32 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:37.879845 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0599e6-231f-4e07-beeb-017a8c6ecf32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:37.898781 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-5d0599e6-231f-4e07-beeb-017a8c6ecf32 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:27:37.900363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-5d0599e6-231f-4e07-beeb-017a8c6ecf32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:27:37.972343 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 26/48] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:27:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:27:37.995524 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2199c581-f2b5-4d5d-ae50-586bc4b386b5 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:38.059948 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:38.103946 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2199c581-f2b5-4d5d-ae50-586bc4b386b5 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:38.137358 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13db2a4-21bf-4e3d-b47e-f3c2cd075db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:38.138158 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13db2a4-21bf-4e3d-b47e-f3c2cd075db4 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:38.138442 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13db2a4-21bf-4e3d-b47e-f3c2cd075db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:38.138719 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13db2a4-21bf-4e3d-b47e-f3c2cd075db4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:38.150470 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13db2a4-21bf-4e3d-b47e-f3c2cd075db4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:27:38.159279 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-d13db2a4-21bf-4e3d-b47e-f3c2cd075db4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:27:38.159279 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:27:38.159279 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:27:38.159279 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:27:38.159279 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:27:38.159279 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:27:38.159279 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:27:38.159279 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:27:38.159279 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:27:38.159279 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:27:38.159279 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:27:38.159279 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:27:38.159279 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:27:38.159279 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:27:38.159279 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:27:38.159279 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:27:38.160882 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:27:38.160882 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:27:38.160882 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:27:38.160882 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:27:38.160882 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:27:38.160882 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:27:38.160882 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:27:38.160882 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:27:38.160882 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:27:38.160882 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:27:38.160882 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:27:38.160882 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:27:38.160882 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:27:38.160882 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:27:38.160882 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:27:38.160882 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:27:38.161885 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:27:38.161885 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:27:38.161885 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:27:38.161885 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:27:38.161885 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:27:38.161885 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:27:38.161885 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:27:38.161885 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:27:38.161885 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:27:38.161885 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:27:38.161885 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:27:38.161885 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:27:38.161885 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 23/49] 10.222.0.36 () {56 vars in 1096 bytes} [Thu May 19 16:27:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:27:38.168504 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0599e6-231f-4e07-beeb-017a8c6ecf32 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:38.268249 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:38.278956 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0599e6-231f-4e07-beeb-017a8c6ecf32 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:38.313949 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93fa817-8fb2-4221-99e4-7e352a1d38de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:38.314769 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93fa817-8fb2-4221-99e4-7e352a1d38de None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:38.315070 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93fa817-8fb2-4221-99e4-7e352a1d38de None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:38.315368 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93fa817-8fb2-4221-99e4-7e352a1d38de None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:38.316152 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93fa817-8fb2-4221-99e4-7e352a1d38de None admin] RBAC: Adding query filter params (name=admin) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:27:38.316477 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93fa817-8fb2-4221-99e4-7e352a1d38de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:38.317974 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93fa817-8fb2-4221-99e4-7e352a1d38de None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:38.327445 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 27/50] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:27:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:38.333523 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13db2a4-21bf-4e3d-b47e-f3c2cd075db4 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:38.383049 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b701810-f38a-4bf6-9c1c-2ce295b9408d None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:38.383347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b701810-f38a-4bf6-9c1c-2ce295b9408d None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:38.383633 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b701810-f38a-4bf6-9c1c-2ce295b9408d None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:38.385035 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 28/51] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:27:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 16:27:38.390519 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f74d5c-9a92-41b6-b134-3413bb7b10c7 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:38.390748 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f74d5c-9a92-41b6-b134-3413bb7b10c7 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:38.391033 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f74d5c-9a92-41b6-b134-3413bb7b10c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:38.408625 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-b5f74d5c-9a92-41b6-b134-3413bb7b10c7 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:27:38.409905 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b5f74d5c-9a92-41b6-b134-3413bb7b10c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:27:38.436679 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13db2a4-21bf-4e3d-b47e-f3c2cd075db4 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:38.457414 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:38.471535 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adfbce6-1101-4900-ab46-fe06c80c262c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:38.472481 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adfbce6-1101-4900-ab46-fe06c80c262c None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:38.473385 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adfbce6-1101-4900-ab46-fe06c80c262c None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:38.473671 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adfbce6-1101-4900-ab46-fe06c80c262c None admin] PATH_INFO: `/v3/users/ac4bfc4d6198473c8d280a14b2c6036f` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:38.475146 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adfbce6-1101-4900-ab46-fe06c80c262c None admin] RBAC: Authorizing `identity:get_user(user_id=ac4bfc4d6198473c8d280a14b2c6036f)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:38.476630 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adfbce6-1101-4900-ab46-fe06c80c262c None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:38.478352 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 24/52] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:27:38 2022] GET /identity/v3/users/ac4bfc4d6198473c8d280a14b2c6036f => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:38.480968 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 29/53] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:27:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 16:27:38.484750 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adfbce6-1101-4900-ab46-fe06c80c262c None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:38.504177 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f74d5c-9a92-41b6-b134-3413bb7b10c7 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:38.593251 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:38.594650 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adfbce6-1101-4900-ab46-fe06c80c262c None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:38.614903 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f74d5c-9a92-41b6-b134-3413bb7b10c7 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:38.632386 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a2814b-638d-4888-b588-f2e676539d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:38.633256 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a2814b-638d-4888-b588-f2e676539d90 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:38.633552 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a2814b-638d-4888-b588-f2e676539d90 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:38.633835 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a2814b-638d-4888-b588-f2e676539d90 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:38.635065 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a2814b-638d-4888-b588-f2e676539d90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:38.636542 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a2814b-638d-4888-b588-f2e676539d90 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:38.638446 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 25/54] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:27:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:27:38.644019 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a2814b-638d-4888-b588-f2e676539d90 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:38.649450 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9973d46d-362f-4135-9e71-3f980394c27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:38.650268 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973d46d-362f-4135-9e71-3f980394c27d None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:38.650554 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973d46d-362f-4135-9e71-3f980394c27d None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:38.650841 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973d46d-362f-4135-9e71-3f980394c27d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:38.652050 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9973d46d-362f-4135-9e71-3f980394c27d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:38.653570 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9973d46d-362f-4135-9e71-3f980394c27d None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:38.655413 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 30/55] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:27:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:27:38.661529 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9973d46d-362f-4135-9e71-3f980394c27d None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:38.703987 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:38.748753 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a2814b-638d-4888-b588-f2e676539d90 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:38.769496 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9973d46d-362f-4135-9e71-3f980394c27d None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:38.792044 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bdad81-6674-4ecc-a0f2-4a42852bf9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:38.792725 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bdad81-6674-4ecc-a0f2-4a42852bf9be None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:38.793020 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bdad81-6674-4ecc-a0f2-4a42852bf9be None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:38.794216 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bdad81-6674-4ecc-a0f2-4a42852bf9be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:38.794216 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bdad81-6674-4ecc-a0f2-4a42852bf9be None admin] RBAC: Adding query filter params (role.id=e2915fe109244a07a9be96bd2b219fdd, scope.domain.id=default, user.id=ac4bfc4d6198473c8d280a14b2c6036f) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:27:38.794525 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bdad81-6674-4ecc-a0f2-4a42852bf9be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:38.796123 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bdad81-6674-4ecc-a0f2-4a42852bf9be None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:38.805944 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 26/56] 10.222.0.36 () {56 vars in 1322 bytes} [Thu May 19 16:27:38 2022] GET /identity/v3/role_assignments?user.id=ac4bfc4d6198473c8d280a14b2c6036f&scope.domain.id=default&role.id=e2915fe109244a07a9be96bd2b219fdd => generated 530 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:27:38.808208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9803c6-affe-47ec-9bb5-09729d3822f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:38.808886 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9803c6-affe-47ec-9bb5-09729d3822f5 None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:38.809200 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9803c6-affe-47ec-9bb5-09729d3822f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:38.809488 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9803c6-affe-47ec-9bb5-09729d3822f5 None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:38.810223 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9803c6-affe-47ec-9bb5-09729d3822f5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:38.811754 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9803c6-affe-47ec-9bb5-09729d3822f5 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:38.839659 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 31/57] 10.222.0.36 () {60 vars in 1140 bytes} [Thu May 19 16:27:38 2022] POST /identity/v3/projects => generated 298 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:27:41.114107 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88285d11-b999-401e-b008-e8c17ac7634a None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:41.114620 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88285d11-b999-401e-b008-e8c17ac7634a None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:41.114708 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88285d11-b999-401e-b008-e8c17ac7634a None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:41.116068 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 27/58] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:27:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:27:41.121640 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3150a16-1ced-4d52-9f3a-83e4d1471ec8 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:41.122068 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3150a16-1ced-4d52-9f3a-83e4d1471ec8 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:41.122208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3150a16-1ced-4d52-9f3a-83e4d1471ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:41.128161 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:41.139349 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-b3150a16-1ced-4d52-9f3a-83e4d1471ec8 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:27:41.140546 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b3150a16-1ced-4d52-9f3a-83e4d1471ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:27:41.205874 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 32/59] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:27:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:27:41.230785 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88285d11-b999-401e-b008-e8c17ac7634a None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:41.333753 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88285d11-b999-401e-b008-e8c17ac7634a None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:41.353582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:41.369645 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8326e17-5737-49b9-bab6-11ae56c14f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:41.370478 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8326e17-5737-49b9-bab6-11ae56c14f41 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:41.370776 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8326e17-5737-49b9-bab6-11ae56c14f41 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:41.371064 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8326e17-5737-49b9-bab6-11ae56c14f41 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:41.372534 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8326e17-5737-49b9-bab6-11ae56c14f41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:41.374079 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8326e17-5737-49b9-bab6-11ae56c14f41 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:41.375784 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 28/60] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:27:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:41.382212 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3150a16-1ced-4d52-9f3a-83e4d1471ec8 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:41.484594 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3150a16-1ced-4d52-9f3a-83e4d1471ec8 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:41.520270 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea198e9-90f0-4212-8e47-08f4a55a619d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:41.520961 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea198e9-90f0-4212-8e47-08f4a55a619d None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:41.521272 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea198e9-90f0-4212-8e47-08f4a55a619d None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:41.521552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea198e9-90f0-4212-8e47-08f4a55a619d None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:41.522574 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea198e9-90f0-4212-8e47-08f4a55a619d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:41.524097 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea198e9-90f0-4212-8e47-08f4a55a619d None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:41.533244 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:41.553452 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 33/61] 10.222.0.36 () {60 vars in 1132 bytes} [Thu May 19 16:27:41 2022] POST /identity/v3/users => generated 275 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:27:44.750175 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06510ebf-371a-45a0-9ee8-456f694c3905 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:44.751210 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06510ebf-371a-45a0-9ee8-456f694c3905 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:44.751742 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06510ebf-371a-45a0-9ee8-456f694c3905 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:44.753438 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 29/62] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:27:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:27:44.758929 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777170e8-ddd6-4845-9441-b9266e43f0d6 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:44.759787 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777170e8-ddd6-4845-9441-b9266e43f0d6 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:44.760273 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777170e8-ddd6-4845-9441-b9266e43f0d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:44.777886 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-777170e8-ddd6-4845-9441-b9266e43f0d6 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:27:44.779379 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-777170e8-ddd6-4845-9441-b9266e43f0d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:27:44.863915 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 34/63] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:27:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:27:44.910597 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06510ebf-371a-45a0-9ee8-456f694c3905 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:45.073637 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06510ebf-371a-45a0-9ee8-456f694c3905 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:45.122859 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c8623c-9c88-48ef-85eb-818463c7c3e0 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:45.123194 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c8623c-9c88-48ef-85eb-818463c7c3e0 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:45.123430 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c8623c-9c88-48ef-85eb-818463c7c3e0 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:45.124845 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 35/64] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:27:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 16:27:45.124953 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca81d3-9568-433d-b764-850541d31dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:45.125863 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca81d3-9568-433d-b764-850541d31dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:45.126255 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca81d3-9568-433d-b764-850541d31dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:45.126547 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca81d3-9568-433d-b764-850541d31dd9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:45.130277 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17782cac-2fc9-4ada-82a6-f7699daacec4 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:45.130572 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17782cac-2fc9-4ada-82a6-f7699daacec4 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:45.130857 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17782cac-2fc9-4ada-82a6-f7699daacec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:45.156276 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bca81d3-9568-433d-b764-850541d31dd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:27:45.158313 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-17782cac-2fc9-4ada-82a6-f7699daacec4 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:27:45.159553 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-17782cac-2fc9-4ada-82a6-f7699daacec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:27:45.169195 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-6bca81d3-9568-433d-b764-850541d31dd9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 16:27:45.169195 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:27:45.169195 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:27:45.169195 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:27:45.169195 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:27:45.169195 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:27:45.169195 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:27:45.169195 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:27:45.169195 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:27:45.169195 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:27:45.169195 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:27:45.169195 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:27:45.169195 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:27:45.169195 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:27:45.169195 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:27:45.169195 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:27:45.170089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:27:45.170089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:27:45.170089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:27:45.170089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:27:45.170089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:27:45.170089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:27:45.170089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:27:45.170089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:27:45.170089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:27:45.170089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:27:45.170089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:27:45.170089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:27:45.170089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:27:45.170089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:27:45.170089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:27:45.170089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:27:45.170863 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:27:45.170863 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:27:45.170863 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:27:45.170863 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:27:45.170863 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:27:45.170863 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:27:45.170863 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:27:45.170863 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:27:45.170863 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:27:45.170863 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:27:45.170863 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 16:27:45.170863 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:27:45.170863 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 30/65] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:27:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:27:45.179475 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca81d3-9568-433d-b764-850541d31dd9 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:45.261716 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 36/66] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:27:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:27:45.263207 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c9ab2-d628-45fd-b0e3-6c00dc146a66 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:45.263514 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c9ab2-d628-45fd-b0e3-6c00dc146a66 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:45.263796 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c9ab2-d628-45fd-b0e3-6c00dc146a66 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:45.265115 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 37/67] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:27:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:27:45.268850 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:45.271203 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903fb499-2f91-4041-bb5c-ae65b7925d44 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:45.271946 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903fb499-2f91-4041-bb5c-ae65b7925d44 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:45.272377 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903fb499-2f91-4041-bb5c-ae65b7925d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:45.290018 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-903fb499-2f91-4041-bb5c-ae65b7925d44 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:27:45.291523 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-903fb499-2f91-4041-bb5c-ae65b7925d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:27:45.325053 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca81d3-9568-433d-b764-850541d31dd9 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:45.342744 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:45.368838 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee29abd-80ab-47f5-87ff-cac10e0bd161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:45.369795 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee29abd-80ab-47f5-87ff-cac10e0bd161 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:45.370234 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee29abd-80ab-47f5-87ff-cac10e0bd161 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:45.370692 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee29abd-80ab-47f5-87ff-cac10e0bd161 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:45.371616 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee29abd-80ab-47f5-87ff-cac10e0bd161 None admin] RBAC: Adding query filter params (name=member) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:27:45.372078 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee29abd-80ab-47f5-87ff-cac10e0bd161 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:45.373685 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee29abd-80ab-47f5-87ff-cac10e0bd161 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:45.375452 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 38/68] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:27:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:27:45.376761 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903fb499-2f91-4041-bb5c-ae65b7925d44 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:45.384195 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 31/69] 10.222.0.36 () {56 vars in 1101 bytes} [Thu May 19 16:27:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:45.398682 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee29abd-80ab-47f5-87ff-cac10e0bd161 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:45.496030 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903fb499-2f91-4041-bb5c-ae65b7925d44 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:45.522891 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee29abd-80ab-47f5-87ff-cac10e0bd161 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:45.536488 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:45.539497 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f406a7c5-0430-4bba-87e6-48e223a58d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:45.541519 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f406a7c5-0430-4bba-87e6-48e223a58d2a None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:45.541665 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f406a7c5-0430-4bba-87e6-48e223a58d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:45.541905 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f406a7c5-0430-4bba-87e6-48e223a58d2a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:45.554637 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f406a7c5-0430-4bba-87e6-48e223a58d2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:27:45.563572 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-f406a7c5-0430-4bba-87e6-48e223a58d2a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:27:45.563572 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:27:45.563572 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:27:45.563572 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:27:45.563572 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:27:45.563572 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:27:45.563572 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:27:45.563572 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:27:45.563572 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:27:45.563572 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:27:45.563572 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:27:45.563572 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:27:45.563572 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:27:45.563572 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:27:45.563572 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:27:45.563572 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:27:45.564652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:27:45.564652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:27:45.564652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:27:45.564652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:27:45.564652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:27:45.564652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:27:45.564652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:27:45.564652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:27:45.564652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:27:45.564652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:27:45.564652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:27:45.564652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:27:45.564652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:27:45.564652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:27:45.564652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:27:45.564652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:27:45.565595 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:27:45.565595 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:27:45.565595 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:27:45.565595 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:27:45.565595 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:27:45.565595 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:27:45.565595 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:27:45.565595 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:27:45.565595 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:27:45.565595 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:27:45.565595 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:27:45.565595 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:27:45.565595 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 39/70] 10.222.0.36 () {56 vars in 1096 bytes} [Thu May 19 16:27:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:27:45.566762 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f406a7c5-0430-4bba-87e6-48e223a58d2a None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:45.567689 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d40ba5-426b-4e2a-91f0-f825a88de3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:45.568795 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d40ba5-426b-4e2a-91f0-f825a88de3ac None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:45.569370 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d40ba5-426b-4e2a-91f0-f825a88de3ac None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:45.569790 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d40ba5-426b-4e2a-91f0-f825a88de3ac None admin] PATH_INFO: `/v3/users/a24c896dd82441dbb4ac8f0fab365a8f` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:45.589358 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d40ba5-426b-4e2a-91f0-f825a88de3ac None admin] RBAC: Authorizing `identity:get_user(user_id=a24c896dd82441dbb4ac8f0fab365a8f)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:45.591183 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d40ba5-426b-4e2a-91f0-f825a88de3ac None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:45.593179 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 32/71] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:27:45 2022] GET /identity/v3/users/a24c896dd82441dbb4ac8f0fab365a8f => generated 275 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:27:45.594209 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d40ba5-426b-4e2a-91f0-f825a88de3ac None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:45.620838 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:45.687798 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f406a7c5-0430-4bba-87e6-48e223a58d2a None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:45.708962 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d40ba5-426b-4e2a-91f0-f825a88de3ac None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:45.728637 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1960b9-4fa4-49bb-9770-b92cf120d4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:45.729494 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1960b9-4fa4-49bb-9770-b92cf120d4e4 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:45.729801 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1960b9-4fa4-49bb-9770-b92cf120d4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:45.730076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1960b9-4fa4-49bb-9770-b92cf120d4e4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:45.740539 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1960b9-4fa4-49bb-9770-b92cf120d4e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:27:45.744137 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4833833-3c97-4bfa-a9f7-541e9b8c70f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:45.744815 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4833833-3c97-4bfa-a9f7-541e9b8c70f2 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:45.745110 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4833833-3c97-4bfa-a9f7-541e9b8c70f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:45.745427 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4833833-3c97-4bfa-a9f7-541e9b8c70f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:45.746197 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4833833-3c97-4bfa-a9f7-541e9b8c70f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:27:45.746513 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4833833-3c97-4bfa-a9f7-541e9b8c70f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:45.747912 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4833833-3c97-4bfa-a9f7-541e9b8c70f2 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:45.748703 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-aa1960b9-4fa4-49bb-9770-b92cf120d4e4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 16:27:45.748703 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:27:45.748703 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:27:45.748703 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:27:45.748703 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:27:45.748703 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:27:45.748703 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:27:45.748703 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:27:45.748703 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:27:45.748703 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:27:45.748703 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:27:45.748703 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:27:45.748703 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:27:45.748703 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:27:45.748703 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:27:45.748703 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:27:45.749554 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:27:45.749554 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:27:45.749554 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:27:45.749554 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:27:45.749554 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:27:45.749554 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:27:45.749554 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:27:45.749554 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:27:45.749554 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:27:45.749554 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:27:45.749554 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:27:45.749554 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:27:45.749554 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:27:45.749554 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:27:45.749554 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:27:45.749554 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:27:45.750294 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:27:45.750294 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:27:45.750294 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:27:45.750294 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:27:45.750294 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:27:45.750294 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:27:45.750294 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:27:45.750294 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:27:45.750294 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:27:45.750294 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:27:45.750294 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 16:27:45.750294 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:27:45.750294 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 40/72] 10.222.0.36 () {56 vars in 1114 bytes} [Thu May 19 16:27:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:27:45.750918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1960b9-4fa4-49bb-9770-b92cf120d4e4 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:45.760596 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 33/73] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:27:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:45.762600 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268db65-fd50-49df-80b9-c52e35072efd None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:45.762903 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268db65-fd50-49df-80b9-c52e35072efd None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:45.763191 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268db65-fd50-49df-80b9-c52e35072efd None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:45.764485 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 34/74] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:27:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:27:45.765857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006f8cd-755c-443b-9b19-c02df98ac5e5 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:45.766152 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006f8cd-755c-443b-9b19-c02df98ac5e5 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:45.766431 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006f8cd-755c-443b-9b19-c02df98ac5e5 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:45.767661 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 35/75] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:27:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:27:45.769668 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f006f8cd-755c-443b-9b19-c02df98ac5e5 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:45.801139 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:45.855212 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:45.858497 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1960b9-4fa4-49bb-9770-b92cf120d4e4 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:45.874652 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f006f8cd-755c-443b-9b19-c02df98ac5e5 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:45.893649 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a6831a-4d84-4f63-9570-3410b4e1c36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:45.894866 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6831a-4d84-4f63-9570-3410b4e1c36f None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:45.895161 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6831a-4d84-4f63-9570-3410b4e1c36f None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:45.895449 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6831a-4d84-4f63-9570-3410b4e1c36f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:45.913113 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9692ff78-a3c6-454d-b6cc-a547f0c76204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:45.913808 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692ff78-a3c6-454d-b6cc-a547f0c76204 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:45.914096 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692ff78-a3c6-454d-b6cc-a547f0c76204 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:45.914373 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692ff78-a3c6-454d-b6cc-a547f0c76204 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:45.915138 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9692ff78-a3c6-454d-b6cc-a547f0c76204 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:27:45.915450 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9692ff78-a3c6-454d-b6cc-a547f0c76204 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:45.916836 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9692ff78-a3c6-454d-b6cc-a547f0c76204 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:45.929667 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 36/76] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:27:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:45.929667 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9692ff78-a3c6-454d-b6cc-a547f0c76204 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:45.980923 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a6831a-4d84-4f63-9570-3410b4e1c36f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:45.983019 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a6831a-4d84-4f63-9570-3410b4e1c36f None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:46.026130 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:46.035644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9692ff78-a3c6-454d-b6cc-a547f0c76204 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:46.069227 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0976935c-6eec-4e37-8a55-ef934cce1d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:46.070116 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0976935c-6eec-4e37-8a55-ef934cce1d1c None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:46.070403 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0976935c-6eec-4e37-8a55-ef934cce1d1c None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:46.070684 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0976935c-6eec-4e37-8a55-ef934cce1d1c None admin] PATH_INFO: `/v3/users/ac4bfc4d6198473c8d280a14b2c6036f` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:46.072237 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0976935c-6eec-4e37-8a55-ef934cce1d1c None admin] RBAC: Authorizing `identity:get_user(user_id=ac4bfc4d6198473c8d280a14b2c6036f)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:46.073761 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0976935c-6eec-4e37-8a55-ef934cce1d1c None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:46.075491 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 37/77] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:27:45 2022] GET /identity/v3/users/ac4bfc4d6198473c8d280a14b2c6036f => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:46.076757 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f906e78-8739-4821-a0cc-22a52698c621 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:46.077051 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f906e78-8739-4821-a0cc-22a52698c621 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:46.077367 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f906e78-8739-4821-a0cc-22a52698c621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:46.082834 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:46.089199 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 41/78] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:27:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 16:27:46.089199 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f6360-931b-4fda-9315-f7541104b999 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:46.089199 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f6360-931b-4fda-9315-f7541104b999 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:46.089666 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f6360-931b-4fda-9315-f7541104b999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:46.095909 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-8f906e78-8739-4821-a0cc-22a52698c621 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:27:46.097175 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-8f906e78-8739-4821-a0cc-22a52698c621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:27:46.106255 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-af7f6360-931b-4fda-9315-f7541104b999 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:27:46.107371 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-af7f6360-931b-4fda-9315-f7541104b999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:27:46.166761 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 38/79] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:27:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:27:46.167652 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f906e78-8739-4821-a0cc-22a52698c621 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:46.183780 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 42/80] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:27:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:27:46.184618 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f6360-931b-4fda-9315-f7541104b999 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:46.278524 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:46.288506 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f906e78-8739-4821-a0cc-22a52698c621 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:46.290024 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f6360-931b-4fda-9315-f7541104b999 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:46.320563 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:46.323316 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba8db54-f425-4163-ba68-0273b371c3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:46.324225 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba8db54-f425-4163-ba68-0273b371c3f1 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:46.324516 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba8db54-f425-4163-ba68-0273b371c3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:46.324799 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba8db54-f425-4163-ba68-0273b371c3f1 None admin] PATH_INFO: `/v3/users/a24c896dd82441dbb4ac8f0fab365a8f` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:46.326071 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba8db54-f425-4163-ba68-0273b371c3f1 None admin] RBAC: Authorizing `identity:get_user(user_id=a24c896dd82441dbb4ac8f0fab365a8f)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:46.327021 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b714a-a716-45ce-92b0-27423238a314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:46.327556 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba8db54-f425-4163-ba68-0273b371c3f1 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:46.327687 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b714a-a716-45ce-92b0-27423238a314 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:46.327979 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b714a-a716-45ce-92b0-27423238a314 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:46.328264 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b714a-a716-45ce-92b0-27423238a314 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:46.329240 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 39/81] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:27:46 2022] GET /identity/v3/users/a24c896dd82441dbb4ac8f0fab365a8f => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:46.329240 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba8db54-f425-4163-ba68-0273b371c3f1 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:46.401621 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8b714a-a716-45ce-92b0-27423238a314 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:46.403140 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8b714a-a716-45ce-92b0-27423238a314 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:46.431453 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba8db54-f425-4163-ba68-0273b371c3f1 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:46.465813 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38b2441-2084-441d-804b-8aaca462925c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:46.466688 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b2441-2084-441d-804b-8aaca462925c None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:46.466982 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b2441-2084-441d-804b-8aaca462925c None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:46.467276 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b2441-2084-441d-804b-8aaca462925c None admin] PATH_INFO: `/v3/projects/d4a456cd7f83414a895ab7982fec2a68` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:46.468495 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38b2441-2084-441d-804b-8aaca462925c None admin] RBAC: Authorizing `identity:get_project(project_id=d4a456cd7f83414a895ab7982fec2a68)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:46.469983 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38b2441-2084-441d-804b-8aaca462925c None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:46.471711 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 40/82] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:27:46 2022] GET /identity/v3/projects/d4a456cd7f83414a895ab7982fec2a68 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:46.472501 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38b2441-2084-441d-804b-8aaca462925c None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:46.500121 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:46.502278 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 43/83] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:27:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 16:27:46.502885 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b714a-a716-45ce-92b0-27423238a314 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:46.536347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:46.576461 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38b2441-2084-441d-804b-8aaca462925c None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:46.607707 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b714a-a716-45ce-92b0-27423238a314 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:46.611914 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f0b1ec-e960-4dbd-b961-4a6b964db113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:46.613278 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0b1ec-e960-4dbd-b961-4a6b964db113 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:46.613587 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0b1ec-e960-4dbd-b961-4a6b964db113 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:46.613873 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0b1ec-e960-4dbd-b961-4a6b964db113 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:46.624556 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f0b1ec-e960-4dbd-b961-4a6b964db113 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:27:46.633051 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-b2f0b1ec-e960-4dbd-b961-4a6b964db113 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 16:27:46.633051 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:27:46.633051 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:27:46.633051 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:27:46.633051 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:27:46.633051 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:27:46.633051 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:27:46.633051 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:27:46.633051 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:27:46.633051 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:27:46.633051 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:27:46.633051 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:27:46.633051 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:27:46.633051 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:27:46.633051 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:27:46.633051 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:27:46.634067 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:27:46.634067 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:27:46.634067 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:27:46.634067 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:27:46.634067 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:27:46.634067 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:27:46.634067 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:27:46.634067 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:27:46.634067 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:27:46.634067 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:27:46.634067 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:27:46.634067 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:27:46.634067 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:27:46.634067 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:27:46.634067 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:27:46.634067 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:27:46.634816 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:27:46.634816 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:27:46.634816 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:27:46.634816 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:27:46.634816 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:27:46.634816 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:27:46.634816 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:27:46.634816 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:27:46.634816 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:27:46.634816 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:27:46.634816 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 16:27:46.634816 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:27:46.634816 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 41/84] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:27:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:27:46.635437 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f0b1ec-e960-4dbd-b961-4a6b964db113 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:46.647123 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf4843-46d9-4c9f-8b4b-118f1fc24b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:46.647932 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf4843-46d9-4c9f-8b4b-118f1fc24b4b None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:46.648218 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf4843-46d9-4c9f-8b4b-118f1fc24b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:46.648499 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf4843-46d9-4c9f-8b4b-118f1fc24b4b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:46.649710 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cf4843-46d9-4c9f-8b4b-118f1fc24b4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:46.651170 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cf4843-46d9-4c9f-8b4b-118f1fc24b4b None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:46.652944 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 44/85] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:27:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:46.653589 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf4843-46d9-4c9f-8b4b-118f1fc24b4b None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:46.721919 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:46.737099 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f0b1ec-e960-4dbd-b961-4a6b964db113 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:46.748952 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:46.752919 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf4843-46d9-4c9f-8b4b-118f1fc24b4b None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:46.771278 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b0a393-346e-44c4-a3e8-ffd2e7d2091d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:46.772197 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b0a393-346e-44c4-a3e8-ffd2e7d2091d None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:46.772562 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b0a393-346e-44c4-a3e8-ffd2e7d2091d None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:46.772785 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b0a393-346e-44c4-a3e8-ffd2e7d2091d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:46.787462 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c2482b-ba06-41f9-94ce-185c0bd7cadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:46.788121 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c2482b-ba06-41f9-94ce-185c0bd7cadc None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:46.788417 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c2482b-ba06-41f9-94ce-185c0bd7cadc None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:46.788701 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c2482b-ba06-41f9-94ce-185c0bd7cadc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:46.789591 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c2482b-ba06-41f9-94ce-185c0bd7cadc None admin] RBAC: Adding query filter params (role.id=0e98fd94a6d24e46b9fe09a22a7df04a, scope.project.id=d4a456cd7f83414a895ab7982fec2a68, user.id=a24c896dd82441dbb4ac8f0fab365a8f) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:27:46.789909 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c2482b-ba06-41f9-94ce-185c0bd7cadc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:46.791413 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c2482b-ba06-41f9-94ce-185c0bd7cadc None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:46.797293 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 45/86] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:27:46 2022] GET /identity/v3/role_assignments?user.id=a24c896dd82441dbb4ac8f0fab365a8f&scope.project.id=d4a456cd7f83414a895ab7982fec2a68&role.id=0e98fd94a6d24e46b9fe09a22a7df04a => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:46.797887 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c2482b-ba06-41f9-94ce-185c0bd7cadc None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:46.854658 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b0a393-346e-44c4-a3e8-ffd2e7d2091d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:46.856195 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b0a393-346e-44c4-a3e8-ffd2e7d2091d None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:46.916981 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c2482b-ba06-41f9-94ce-185c0bd7cadc None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:46.955965 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfaf8aa-c331-4089-9ab2-42f34f92e068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:46.956975 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfaf8aa-c331-4089-9ab2-42f34f92e068 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:46.957296 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfaf8aa-c331-4089-9ab2-42f34f92e068 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:46.957579 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfaf8aa-c331-4089-9ab2-42f34f92e068 None admin] PATH_INFO: `/v3/projects/d4a456cd7f83414a895ab7982fec2a68` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:46.958865 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfaf8aa-c331-4089-9ab2-42f34f92e068 None admin] RBAC: Authorizing `identity:get_project(project_id=d4a456cd7f83414a895ab7982fec2a68)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:46.960389 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfaf8aa-c331-4089-9ab2-42f34f92e068 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:46.962235 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 46/87] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:27:46 2022] GET /identity/v3/projects/d4a456cd7f83414a895ab7982fec2a68 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:46.963043 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfaf8aa-c331-4089-9ab2-42f34f92e068 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:46.977985 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:46.999789 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:47.006965 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 42/88] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:27:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 16:27:47.007821 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b0a393-346e-44c4-a3e8-ffd2e7d2091d None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:47.077317 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfaf8aa-c331-4089-9ab2-42f34f92e068 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:47.119407 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d51943-7254-44ad-b70c-435898df3ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:47.120091 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d51943-7254-44ad-b70c-435898df3ee7 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:47.120381 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d51943-7254-44ad-b70c-435898df3ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:47.120665 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d51943-7254-44ad-b70c-435898df3ee7 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:47.121472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d51943-7254-44ad-b70c-435898df3ee7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:27:47.121786 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d51943-7254-44ad-b70c-435898df3ee7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:47.123213 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d51943-7254-44ad-b70c-435898df3ee7 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:47.128021 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b0a393-346e-44c4-a3e8-ffd2e7d2091d None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:47.137234 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 47/89] 10.222.0.36 () {56 vars in 1101 bytes} [Thu May 19 16:27:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:47.137234 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d51943-7254-44ad-b70c-435898df3ee7 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:47.167532 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1ec6f0-882d-4c5a-89fd-532f940fbc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:47.168126 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1ec6f0-882d-4c5a-89fd-532f940fbc93 None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:47.168415 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1ec6f0-882d-4c5a-89fd-532f940fbc93 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:47.168694 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1ec6f0-882d-4c5a-89fd-532f940fbc93 None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:47.169570 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1ec6f0-882d-4c5a-89fd-532f940fbc93 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:47.171157 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1ec6f0-882d-4c5a-89fd-532f940fbc93 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:47.209198 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 43/90] 10.222.0.36 () {60 vars in 1132 bytes} [Thu May 19 16:27:47 2022] POST /identity/v3/users => generated 289 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:27:47.209198 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1ec6f0-882d-4c5a-89fd-532f940fbc93 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:47.251181 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:47.266566 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d51943-7254-44ad-b70c-435898df3ee7 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:47.267522 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:47.315892 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebf8563-a5d1-4129-81d2-2869b459ab22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:47.317126 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf8563-a5d1-4129-81d2-2869b459ab22 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:47.317449 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf8563-a5d1-4129-81d2-2869b459ab22 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:47.317734 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf8563-a5d1-4129-81d2-2869b459ab22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:47.318659 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebf8563-a5d1-4129-81d2-2869b459ab22 None admin] RBAC: Adding query filter params (role.id=e2915fe109244a07a9be96bd2b219fdd, scope.project.id=d4a456cd7f83414a895ab7982fec2a68, user.id=ac4bfc4d6198473c8d280a14b2c6036f) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:27:47.318983 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebf8563-a5d1-4129-81d2-2869b459ab22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:47.320505 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebf8563-a5d1-4129-81d2-2869b459ab22 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:47.333194 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 48/91] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:27:47 2022] GET /identity/v3/role_assignments?user.id=ac4bfc4d6198473c8d280a14b2c6036f&scope.project.id=d4a456cd7f83414a895ab7982fec2a68&role.id=e2915fe109244a07a9be96bd2b219fdd => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:47.333194 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebf8563-a5d1-4129-81d2-2869b459ab22 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:47.360568 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1ec6f0-882d-4c5a-89fd-532f940fbc93 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:47.409089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e25170-0c57-4073-8f9d-46e03acc6f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:47.410062 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e25170-0c57-4073-8f9d-46e03acc6f3d None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:47.411065 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e25170-0c57-4073-8f9d-46e03acc6f3d None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:47.411355 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e25170-0c57-4073-8f9d-46e03acc6f3d None admin] PATH_INFO: `/v3/projects/d4a456cd7f83414a895ab7982fec2a68` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:47.412912 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e25170-0c57-4073-8f9d-46e03acc6f3d None admin] RBAC: Authorizing `identity:get_project(project_id=d4a456cd7f83414a895ab7982fec2a68)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:47.414568 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e25170-0c57-4073-8f9d-46e03acc6f3d None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:47.416503 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 44/92] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:27:47 2022] GET /identity/v3/projects/d4a456cd7f83414a895ab7982fec2a68 => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:47.423312 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e25170-0c57-4073-8f9d-46e03acc6f3d None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:47.547202 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebf8563-a5d1-4129-81d2-2869b459ab22 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:47.582147 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e25170-0c57-4073-8f9d-46e03acc6f3d None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:47.596908 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:47.605413 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d5315e-c91f-47cf-942d-b672affa1784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:47.607297 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5315e-c91f-47cf-942d-b672affa1784 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:47.607849 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5315e-c91f-47cf-942d-b672affa1784 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:47.608346 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5315e-c91f-47cf-942d-b672affa1784 None admin] PATH_INFO: `/v3/users/a24c896dd82441dbb4ac8f0fab365a8f` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:47.609930 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d5315e-c91f-47cf-942d-b672affa1784 None admin] RBAC: Authorizing `identity:get_user(user_id=a24c896dd82441dbb4ac8f0fab365a8f)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:47.612480 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d5315e-c91f-47cf-942d-b672affa1784 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:47.615977 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 49/93] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:27:47 2022] GET /identity/v3/users/a24c896dd82441dbb4ac8f0fab365a8f => generated 275 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:47.621943 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d5315e-c91f-47cf-942d-b672affa1784 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:47.625761 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26e0f19-b59e-49ba-82ca-43f273cd3ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:47.626645 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e0f19-b59e-49ba-82ca-43f273cd3ca1 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:47.627008 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e0f19-b59e-49ba-82ca-43f273cd3ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:47.627367 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e0f19-b59e-49ba-82ca-43f273cd3ca1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:47.628327 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26e0f19-b59e-49ba-82ca-43f273cd3ca1 None admin] RBAC: Adding query filter params (role.id=50f62a33eb434682baa7529619fe14b0, scope.project.id=d4a456cd7f83414a895ab7982fec2a68, user.id=a24c896dd82441dbb4ac8f0fab365a8f) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:27:47.628728 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26e0f19-b59e-49ba-82ca-43f273cd3ca1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:47.630315 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26e0f19-b59e-49ba-82ca-43f273cd3ca1 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:47.636870 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 45/94] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:27:47 2022] GET /identity/v3/role_assignments?user.id=a24c896dd82441dbb4ac8f0fab365a8f&scope.project.id=d4a456cd7f83414a895ab7982fec2a68&role.id=50f62a33eb434682baa7529619fe14b0 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:27:47.653651 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:47.774251 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d5315e-c91f-47cf-942d-b672affa1784 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:47.829866 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb4e2d-07ca-4793-9850-b6fa54fa621d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:47.830575 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb4e2d-07ca-4793-9850-b6fa54fa621d None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:47.830873 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb4e2d-07ca-4793-9850-b6fa54fa621d None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:47.832454 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb4e2d-07ca-4793-9850-b6fa54fa621d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:47.911397 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eb4e2d-07ca-4793-9850-b6fa54fa621d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:47.912958 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eb4e2d-07ca-4793-9850-b6fa54fa621d None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:47.944977 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:48.019744 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 50/95] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:27:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 16:27:48.025756 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26e0f19-b59e-49ba-82ca-43f273cd3ca1 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:48.167112 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26e0f19-b59e-49ba-82ca-43f273cd3ca1 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:48.205746 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde3148-9c75-4734-aec3-b2acff95679c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:48.207610 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde3148-9c75-4734-aec3-b2acff95679c None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:48.208010 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde3148-9c75-4734-aec3-b2acff95679c None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:48.208319 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde3148-9c75-4734-aec3-b2acff95679c None admin] PATH_INFO: `/v3/projects/e2a683cd4fca4fe5ab8a9837a71d5988` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:48.209755 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fde3148-9c75-4734-aec3-b2acff95679c None admin] RBAC: Authorizing `identity:get_project(project_id=e2a683cd4fca4fe5ab8a9837a71d5988)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:48.211243 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fde3148-9c75-4734-aec3-b2acff95679c None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:48.213244 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 46/96] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:27:48 2022] GET /identity/v3/projects/e2a683cd4fca4fe5ab8a9837a71d5988 => generated 312 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:48.222889 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb4e2d-07ca-4793-9850-b6fa54fa621d None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:48.382566 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb4e2d-07ca-4793-9850-b6fa54fa621d None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:48.410958 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:48.422436 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee22eb86-ebed-4019-8073-0671db3940af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:48.423119 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22eb86-ebed-4019-8073-0671db3940af None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:48.423410 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22eb86-ebed-4019-8073-0671db3940af None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:48.423694 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22eb86-ebed-4019-8073-0671db3940af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:48.424550 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee22eb86-ebed-4019-8073-0671db3940af None admin] RBAC: Adding query filter params (role.id=0e98fd94a6d24e46b9fe09a22a7df04a, scope.project.id=e2a683cd4fca4fe5ab8a9837a71d5988, user.id=a24c896dd82441dbb4ac8f0fab365a8f) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:27:48.424872 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee22eb86-ebed-4019-8073-0671db3940af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:48.426432 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee22eb86-ebed-4019-8073-0671db3940af None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:48.432532 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 51/97] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:27:48 2022] GET /identity/v3/role_assignments?user.id=a24c896dd82441dbb4ac8f0fab365a8f&scope.project.id=e2a683cd4fca4fe5ab8a9837a71d5988&role.id=0e98fd94a6d24e46b9fe09a22a7df04a => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:49.705668 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da0bc2-f7bc-4e06-9d2a-7ed9184dd872 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:49.706944 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da0bc2-f7bc-4e06-9d2a-7ed9184dd872 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:49.706944 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da0bc2-f7bc-4e06-9d2a-7ed9184dd872 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:49.708302 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 47/98] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:27:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:27:49.714127 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d0b1e-7dbf-48b1-befb-0a53d2b18a1f None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:49.714629 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d0b1e-7dbf-48b1-befb-0a53d2b18a1f None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:49.714933 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d0b1e-7dbf-48b1-befb-0a53d2b18a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:49.732982 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-376d0b1e-7dbf-48b1-befb-0a53d2b18a1f None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:27:49.734359 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-376d0b1e-7dbf-48b1-befb-0a53d2b18a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:27:49.816733 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 52/99] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:27:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 16:27:49.839573 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da0bc2-f7bc-4e06-9d2a-7ed9184dd872 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:49.898977 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:49.986765 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da0bc2-f7bc-4e06-9d2a-7ed9184dd872 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:50.045543 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c163ed8-10fa-4d35-bd10-b32b5c74d3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:50.046381 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c163ed8-10fa-4d35-bd10-b32b5c74d3fc None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:50.046669 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c163ed8-10fa-4d35-bd10-b32b5c74d3fc None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:50.046952 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c163ed8-10fa-4d35-bd10-b32b5c74d3fc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:50.067168 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c163ed8-10fa-4d35-bd10-b32b5c74d3fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:27:50.077048 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-3c163ed8-10fa-4d35-bd10-b32b5c74d3fc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 16:27:50.077048 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:27:50.077048 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:27:50.077048 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:27:50.077048 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:27:50.077048 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:27:50.077048 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:27:50.077048 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:27:50.077048 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:27:50.077048 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:27:50.077048 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:27:50.077048 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:27:50.077048 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:27:50.077048 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:27:50.077048 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:27:50.077048 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:27:50.078070 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:27:50.078070 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:27:50.078070 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:27:50.078070 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:27:50.078070 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:27:50.078070 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:27:50.078070 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:27:50.078070 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:27:50.078070 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:27:50.078070 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:27:50.078070 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:27:50.078070 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:27:50.078070 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:27:50.078070 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:27:50.078070 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:27:50.078070 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:27:50.078881 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:27:50.078881 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:27:50.078881 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:27:50.078881 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:27:50.078881 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:27:50.078881 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:27:50.078881 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:27:50.078881 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:27:50.078881 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:27:50.078881 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:27:50.078881 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 16:27:50.078881 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:27:50.078881 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 48/100] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:27:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:27:50.086646 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376d0b1e-7dbf-48b1-befb-0a53d2b18a1f None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:50.225502 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376d0b1e-7dbf-48b1-befb-0a53d2b18a1f None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:50.248388 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:50.275305 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c939ea6-f901-4db8-baa8-0290776e2beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:50.276049 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c939ea6-f901-4db8-baa8-0290776e2beb None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:50.276342 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c939ea6-f901-4db8-baa8-0290776e2beb None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:50.276624 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c939ea6-f901-4db8-baa8-0290776e2beb None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:50.277421 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c939ea6-f901-4db8-baa8-0290776e2beb None admin] RBAC: Adding query filter params (name=member) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:27:50.277735 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c939ea6-f901-4db8-baa8-0290776e2beb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:50.279155 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c939ea6-f901-4db8-baa8-0290776e2beb None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:50.293220 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 53/101] 10.222.0.36 () {56 vars in 1101 bytes} [Thu May 19 16:27:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:50.298690 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c163ed8-10fa-4d35-bd10-b32b5c74d3fc None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:50.331660 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e306f00c-73b0-426a-8018-9dc7ac6b7708 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:50.332108 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e306f00c-73b0-426a-8018-9dc7ac6b7708 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:50.332404 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e306f00c-73b0-426a-8018-9dc7ac6b7708 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:50.334531 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 54/102] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:27:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 16:27:50.339738 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2a56f-0b1b-4b58-9f7e-e22458190292 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:50.340038 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2a56f-0b1b-4b58-9f7e-e22458190292 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:50.340329 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2a56f-0b1b-4b58-9f7e-e22458190292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:50.360961 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-1ad2a56f-0b1b-4b58-9f7e-e22458190292 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:27:50.362408 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-1ad2a56f-0b1b-4b58-9f7e-e22458190292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:27:50.466335 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c163ed8-10fa-4d35-bd10-b32b5c74d3fc None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:50.468150 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:50.515709 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 55/103] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:27:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 16:27:50.553593 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568a9c7-4af4-4e2f-94c2-090116626945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:50.554511 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568a9c7-4af4-4e2f-94c2-090116626945 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:50.554792 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad2a56f-0b1b-4b58-9f7e-e22458190292 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:50.554916 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568a9c7-4af4-4e2f-94c2-090116626945 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:50.555076 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568a9c7-4af4-4e2f-94c2-090116626945 None admin] PATH_INFO: `/v3/users/a24c896dd82441dbb4ac8f0fab365a8f` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:50.556376 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a568a9c7-4af4-4e2f-94c2-090116626945 None admin] RBAC: Authorizing `identity:get_user(user_id=a24c896dd82441dbb4ac8f0fab365a8f)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:50.557910 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a568a9c7-4af4-4e2f-94c2-090116626945 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:50.559638 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 49/104] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:27:50 2022] GET /identity/v3/users/a24c896dd82441dbb4ac8f0fab365a8f => generated 275 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:50.565016 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568a9c7-4af4-4e2f-94c2-090116626945 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:50.776582 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad2a56f-0b1b-4b58-9f7e-e22458190292 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:50.797574 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:50.798213 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568a9c7-4af4-4e2f-94c2-090116626945 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:50.863672 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd50d7-604b-419a-9dc9-83cd30ea0cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:50.864569 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd50d7-604b-419a-9dc9-83cd30ea0cad None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:50.864887 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd50d7-604b-419a-9dc9-83cd30ea0cad None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:50.865239 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd50d7-604b-419a-9dc9-83cd30ea0cad None admin] PATH_INFO: `/v3/projects/d4a456cd7f83414a895ab7982fec2a68` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:50.866461 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad3c8e-db20-4f8b-9a89-3d1ca563cd58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:50.866582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fd50d7-604b-419a-9dc9-83cd30ea0cad None admin] RBAC: Authorizing `identity:get_project(project_id=d4a456cd7f83414a895ab7982fec2a68)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:50.867290 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad3c8e-db20-4f8b-9a89-3d1ca563cd58 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:50.867580 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad3c8e-db20-4f8b-9a89-3d1ca563cd58 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:50.867864 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad3c8e-db20-4f8b-9a89-3d1ca563cd58 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:50.868057 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fd50d7-604b-419a-9dc9-83cd30ea0cad None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:50.870032 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 50/105] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:27:50 2022] GET /identity/v3/projects/d4a456cd7f83414a895ab7982fec2a68 => generated 298 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:50.878731 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd50d7-604b-419a-9dc9-83cd30ea0cad None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:50.910496 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ad3c8e-db20-4f8b-9a89-3d1ca563cd58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:27:50.918797 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-13ad3c8e-db20-4f8b-9a89-3d1ca563cd58 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:27:50.918797 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:27:50.918797 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:27:50.918797 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:27:50.918797 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:27:50.918797 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:27:50.918797 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:27:50.918797 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:27:50.918797 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:27:50.918797 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:27:50.918797 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:27:50.918797 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:27:50.918797 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:27:50.918797 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:27:50.918797 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:27:50.918797 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:27:50.919722 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:27:50.919722 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:27:50.919722 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:27:50.919722 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:27:50.919722 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:27:50.919722 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:27:50.919722 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:27:50.919722 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:27:50.919722 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:27:50.919722 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:27:50.919722 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:27:50.919722 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:27:50.919722 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:27:50.919722 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:27:50.919722 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:27:50.919722 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:27:50.920562 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:27:50.920562 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:27:50.920562 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:27:50.920562 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:27:50.920562 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:27:50.920562 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:27:50.920562 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:27:50.920562 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:27:50.920562 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:27:50.920562 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:27:50.920562 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:27:50.920562 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:27:50.920562 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 56/106] 10.222.0.36 () {56 vars in 1096 bytes} [Thu May 19 16:27:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 16:27:50.929476 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43fa8f4-8586-4d7a-ae4c-15df8e84b1b7 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:50.930266 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43fa8f4-8586-4d7a-ae4c-15df8e84b1b7 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:50.930748 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43fa8f4-8586-4d7a-ae4c-15df8e84b1b7 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:50.932489 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 57/107] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:27:50 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:27:50.940419 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43fa8f4-8586-4d7a-ae4c-15df8e84b1b7 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:50.978580 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:51.102729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd50d7-604b-419a-9dc9-83cd30ea0cad None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:51.149969 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43fa8f4-8586-4d7a-ae4c-15df8e84b1b7 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:51.158762 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255f4a06-9966-4afa-bd8e-aecb82dd6f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:51.160112 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f4a06-9966-4afa-bd8e-aecb82dd6f79 None admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:51.160748 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f4a06-9966-4afa-bd8e-aecb82dd6f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:51.161274 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f4a06-9966-4afa-bd8e-aecb82dd6f79 None admin] PATH_INFO: `/v3/projects/d4a456cd7f83414a895ab7982fec2a68/users/a24c896dd82441dbb4ac8f0fab365a8f/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:51.163940 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255f4a06-9966-4afa-bd8e-aecb82dd6f79 None admin] RBAC: Authorizing `identity:create_grant(project_id=d4a456cd7f83414a895ab7982fec2a68, user_id=a24c896dd82441dbb4ac8f0fab365a8f, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:51.165893 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255f4a06-9966-4afa-bd8e-aecb82dd6f79 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:51.195925 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 51/108] 10.222.0.36 () {58 vars in 1439 bytes} [Thu May 19 16:27:50 2022] PUT /identity/v3/projects/d4a456cd7f83414a895ab7982fec2a68/users/a24c896dd82441dbb4ac8f0fab365a8f/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:27:51.199328 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee79c503-5b8b-48f9-9576-ed5e851b20de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:51.200064 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360105b2-02df-4db0-9052-a6d972f3fdb4 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:51.201191 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360105b2-02df-4db0-9052-a6d972f3fdb4 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:51.201636 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79c503-5b8b-48f9-9576-ed5e851b20de None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:51.202242 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360105b2-02df-4db0-9052-a6d972f3fdb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:51.202753 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79c503-5b8b-48f9-9576-ed5e851b20de None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:51.204276 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79c503-5b8b-48f9-9576-ed5e851b20de None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:51.206238 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee79c503-5b8b-48f9-9576-ed5e851b20de None admin] RBAC: Adding query filter params (name=admin) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:27:51.207167 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee79c503-5b8b-48f9-9576-ed5e851b20de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:51.211212 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee79c503-5b8b-48f9-9576-ed5e851b20de None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:51.232072 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 58/109] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:27:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:51.239112 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee79c503-5b8b-48f9-9576-ed5e851b20de None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:51.242679 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-360105b2-02df-4db0-9052-a6d972f3fdb4 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:27:51.244885 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-360105b2-02df-4db0-9052-a6d972f3fdb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:27:51.294352 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:51.390425 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee79c503-5b8b-48f9-9576-ed5e851b20de None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:51.412106 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 52/110] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:27:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:27:51.412106 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e1a6e9-f20b-479c-96e4-cb78845cc19c None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:51.412106 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e1a6e9-f20b-479c-96e4-cb78845cc19c None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:51.412106 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e1a6e9-f20b-479c-96e4-cb78845cc19c None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:51.413305 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 53/111] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:27:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:27:51.415257 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d647e7-8373-49dc-86ac-42a86724d0d0 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:51.415592 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d647e7-8373-49dc-86ac-42a86724d0d0 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:51.415900 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d647e7-8373-49dc-86ac-42a86724d0d0 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:51.417753 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 54/112] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:27:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:27:51.418936 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06695e61-b5c6-44a4-b0fa-e66eb4193947 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:51.419266 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06695e61-b5c6-44a4-b0fa-e66eb4193947 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:51.419582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06695e61-b5c6-44a4-b0fa-e66eb4193947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:51.458669 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64986de8-504d-48fe-a63d-74ad6310d7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:51.460011 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64986de8-504d-48fe-a63d-74ad6310d7ba None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:51.460507 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64986de8-504d-48fe-a63d-74ad6310d7ba None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:51.461023 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64986de8-504d-48fe-a63d-74ad6310d7ba None admin] PATH_INFO: `/v3/users/ac4bfc4d6198473c8d280a14b2c6036f` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:51.462768 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64986de8-504d-48fe-a63d-74ad6310d7ba None admin] RBAC: Authorizing `identity:get_user(user_id=ac4bfc4d6198473c8d280a14b2c6036f)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:51.463032 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-06695e61-b5c6-44a4-b0fa-e66eb4193947 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:27:51.464263 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-06695e61-b5c6-44a4-b0fa-e66eb4193947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:27:51.465137 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64986de8-504d-48fe-a63d-74ad6310d7ba None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:51.467236 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 59/113] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:27:51 2022] GET /identity/v3/users/ac4bfc4d6198473c8d280a14b2c6036f => generated 247 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:51.468935 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c47fce6-dfca-4cfa-ba91-2f985c17ce12 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:51.469442 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c47fce6-dfca-4cfa-ba91-2f985c17ce12 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:51.469912 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c47fce6-dfca-4cfa-ba91-2f985c17ce12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:51.489412 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-3c47fce6-dfca-4cfa-ba91-2f985c17ce12 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:27:51.491334 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-3c47fce6-dfca-4cfa-ba91-2f985c17ce12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:27:51.500266 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:51.552535 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 55/114] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:27:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:27:51.554043 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06695e61-b5c6-44a4-b0fa-e66eb4193947 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:51.592176 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 60/115] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:27:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:27:51.593044 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c47fce6-dfca-4cfa-ba91-2f985c17ce12 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:51.639126 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:51.703800 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c47fce6-dfca-4cfa-ba91-2f985c17ce12 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:51.709809 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06695e61-b5c6-44a4-b0fa-e66eb4193947 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:51.740462 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6945eef-eb5e-44c3-a06f-6b4fd256c8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:51.741364 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6945eef-eb5e-44c3-a06f-6b4fd256c8a2 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:51.741650 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6945eef-eb5e-44c3-a06f-6b4fd256c8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:51.741931 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6945eef-eb5e-44c3-a06f-6b4fd256c8a2 None admin] PATH_INFO: `/v3/projects/d4a456cd7f83414a895ab7982fec2a68` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:51.743136 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6945eef-eb5e-44c3-a06f-6b4fd256c8a2 None admin] RBAC: Authorizing `identity:get_project(project_id=d4a456cd7f83414a895ab7982fec2a68)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:51.744607 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6945eef-eb5e-44c3-a06f-6b4fd256c8a2 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:51.746666 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 61/116] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:27:51 2022] GET /identity/v3/projects/d4a456cd7f83414a895ab7982fec2a68 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:51.747512 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6945eef-eb5e-44c3-a06f-6b4fd256c8a2 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:51.756873 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c59d553-9db1-485c-b8df-ddfe092ba4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:51.757715 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59d553-9db1-485c-b8df-ddfe092ba4b8 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:51.758001 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59d553-9db1-485c-b8df-ddfe092ba4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:51.758282 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59d553-9db1-485c-b8df-ddfe092ba4b8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:51.769800 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c59d553-9db1-485c-b8df-ddfe092ba4b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:27:51.778153 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-6c59d553-9db1-485c-b8df-ddfe092ba4b8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 16:27:51.778153 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:27:51.778153 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:27:51.778153 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:27:51.778153 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:27:51.778153 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:27:51.778153 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:27:51.778153 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:27:51.778153 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:27:51.778153 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:27:51.778153 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:27:51.778153 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:27:51.778153 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:27:51.778153 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:27:51.778153 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:27:51.778153 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:27:51.779029 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:27:51.779029 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:27:51.779029 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:27:51.779029 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:27:51.779029 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:27:51.779029 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:27:51.779029 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:27:51.779029 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:27:51.779029 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:27:51.779029 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:27:51.779029 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:27:51.779029 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:27:51.779029 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:27:51.779029 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:27:51.779029 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:27:51.779029 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:27:51.780322 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:27:51.780322 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:27:51.780322 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:27:51.780322 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:27:51.780322 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:27:51.780322 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:27:51.780322 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:27:51.780322 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:27:51.780322 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:27:51.780322 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:27:51.780322 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 16:27:51.780322 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:27:51.780322 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 56/117] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:27:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:27:51.780973 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c59d553-9db1-485c-b8df-ddfe092ba4b8 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:51.805625 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:51.869463 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:51.873088 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6945eef-eb5e-44c3-a06f-6b4fd256c8a2 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:51.903984 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c59d553-9db1-485c-b8df-ddfe092ba4b8 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:51.918753 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f729ef2-c9cc-45af-9b50-74347f87e68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:51.919576 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f729ef2-c9cc-45af-9b50-74347f87e68d None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:51.920003 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f729ef2-c9cc-45af-9b50-74347f87e68d None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:51.920247 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f729ef2-c9cc-45af-9b50-74347f87e68d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:51.924160 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f729ef2-c9cc-45af-9b50-74347f87e68d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:51.924160 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f729ef2-c9cc-45af-9b50-74347f87e68d None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:51.927904 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 62/118] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:27:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:51.928621 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f729ef2-c9cc-45af-9b50-74347f87e68d None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:51.951055 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb85600-72a6-425c-a618-28b8083c93e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:51.951055 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb85600-72a6-425c-a618-28b8083c93e9 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:51.951055 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb85600-72a6-425c-a618-28b8083c93e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:51.951055 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb85600-72a6-425c-a618-28b8083c93e9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:51.961664 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb85600-72a6-425c-a618-28b8083c93e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:27:51.970161 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-0fb85600-72a6-425c-a618-28b8083c93e9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 16:27:51.970161 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:27:51.970161 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:27:51.970161 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:27:51.970161 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:27:51.970161 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:27:51.970161 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:27:51.970161 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:27:51.970161 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:27:51.970161 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:27:51.970161 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:27:51.970161 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:27:51.970161 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:27:51.970161 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:27:51.970161 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:27:51.970161 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:27:51.971030 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:27:51.971030 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:27:51.971030 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:27:51.971030 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:27:51.971030 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:27:51.971030 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:27:51.971030 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:27:51.971030 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:27:51.971030 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:27:51.971030 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:27:51.971030 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:27:51.971030 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:27:51.971030 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:27:51.971030 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:27:51.971030 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:27:51.971030 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:27:51.971780 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:27:51.971780 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:27:51.971780 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:27:51.971780 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:27:51.971780 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:27:51.971780 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:27:51.971780 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:27:51.971780 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:27:51.971780 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:27:51.971780 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:27:51.971780 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 16:27:51.971780 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:27:51.971780 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 57/119] 10.222.0.36 () {56 vars in 1114 bytes} [Thu May 19 16:27:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:27:51.973001 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb85600-72a6-425c-a618-28b8083c93e9 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:52.049286 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f729ef2-c9cc-45af-9b50-74347f87e68d None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:52.057812 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:52.084593 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb85600-72a6-425c-a618-28b8083c93e9 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:52.086260 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3a1ce-77bc-4a3e-8e98-530adb0fb31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:52.087178 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3a1ce-77bc-4a3e-8e98-530adb0fb31f None admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:52.087461 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3a1ce-77bc-4a3e-8e98-530adb0fb31f None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:52.087741 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3a1ce-77bc-4a3e-8e98-530adb0fb31f None admin] PATH_INFO: `/v3/projects/d4a456cd7f83414a895ab7982fec2a68/users/ac4bfc4d6198473c8d280a14b2c6036f/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:52.089751 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe3a1ce-77bc-4a3e-8e98-530adb0fb31f None admin] RBAC: Authorizing `identity:create_grant(project_id=d4a456cd7f83414a895ab7982fec2a68, user_id=ac4bfc4d6198473c8d280a14b2c6036f, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:52.091253 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe3a1ce-77bc-4a3e-8e98-530adb0fb31f None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:52.116225 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 63/120] 10.222.0.36 () {58 vars in 1439 bytes} [Thu May 19 16:27:51 2022] PUT /identity/v3/projects/d4a456cd7f83414a895ab7982fec2a68/users/ac4bfc4d6198473c8d280a14b2c6036f/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:27:52.116225 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3a1ce-77bc-4a3e-8e98-530adb0fb31f None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:52.122534 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87337e4b-fb0b-4396-88dd-a5f68a065e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:52.123226 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87337e4b-fb0b-4396-88dd-a5f68a065e79 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:52.123514 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87337e4b-fb0b-4396-88dd-a5f68a065e79 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:52.123796 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87337e4b-fb0b-4396-88dd-a5f68a065e79 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:52.124637 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87337e4b-fb0b-4396-88dd-a5f68a065e79 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:27:52.125024 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87337e4b-fb0b-4396-88dd-a5f68a065e79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:52.126571 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87337e4b-fb0b-4396-88dd-a5f68a065e79 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:52.137022 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 58/121] 10.222.0.36 () {56 vars in 1101 bytes} [Thu May 19 16:27:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:52.138011 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87337e4b-fb0b-4396-88dd-a5f68a065e79 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:52.155787 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:52.263607 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3a1ce-77bc-4a3e-8e98-530adb0fb31f None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:52.274714 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87337e4b-fb0b-4396-88dd-a5f68a065e79 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:52.315389 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d8779-4ab2-4c99-926f-9e0cdc2c9932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:52.316657 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d8779-4ab2-4c99-926f-9e0cdc2c9932 None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:52.316946 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d8779-4ab2-4c99-926f-9e0cdc2c9932 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:52.317247 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d8779-4ab2-4c99-926f-9e0cdc2c9932 None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:52.317985 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502d8779-4ab2-4c99-926f-9e0cdc2c9932 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:52.320164 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502d8779-4ab2-4c99-926f-9e0cdc2c9932 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:52.324123 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc8d749-2f14-4902-a96e-55f0504ab793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:52.324973 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc8d749-2f14-4902-a96e-55f0504ab793 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:52.325395 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc8d749-2f14-4902-a96e-55f0504ab793 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:52.325682 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc8d749-2f14-4902-a96e-55f0504ab793 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:52.326454 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc8d749-2f14-4902-a96e-55f0504ab793 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:27:52.326769 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc8d749-2f14-4902-a96e-55f0504ab793 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:52.328210 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc8d749-2f14-4902-a96e-55f0504ab793 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:52.338180 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 59/122] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:27:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:52.339524 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad295539-ca32-469b-863d-0a0436750d34 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:52.339822 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad295539-ca32-469b-863d-0a0436750d34 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:52.340447 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad295539-ca32-469b-863d-0a0436750d34 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:52.345485 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 60/123] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:27:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:27:52.345485 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad295539-ca32-469b-863d-0a0436750d34 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:52.345917 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:52.359622 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 64/124] 10.222.0.36 () {60 vars in 1141 bytes} [Thu May 19 16:27:52 2022] POST /identity/v3/projects => generated 302 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:27:52.360490 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d8779-4ab2-4c99-926f-9e0cdc2c9932 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:52.468406 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad295539-ca32-469b-863d-0a0436750d34 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:52.509893 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ec5398-1b47-4545-8c49-acbb36e8c932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:52.511209 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec5398-1b47-4545-8c49-acbb36e8c932 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:52.511724 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec5398-1b47-4545-8c49-acbb36e8c932 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:52.512222 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec5398-1b47-4545-8c49-acbb36e8c932 None admin] PATH_INFO: `/v3/users/16bccb2c02f64fb0892b8adc61acbd33` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:52.523967 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:52.531808 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ec5398-1b47-4545-8c49-acbb36e8c932 None admin] RBAC: Authorizing `identity:get_user(user_id=16bccb2c02f64fb0892b8adc61acbd33)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:52.533791 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ec5398-1b47-4545-8c49-acbb36e8c932 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:52.536520 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 61/125] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:27:52 2022] GET /identity/v3/users/16bccb2c02f64fb0892b8adc61acbd33 => generated 289 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:52.538162 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5159ac0-9481-4406-beaa-e237932a7fb8 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:52.538833 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5159ac0-9481-4406-beaa-e237932a7fb8 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:52.539227 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5159ac0-9481-4406-beaa-e237932a7fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:52.560585 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-e5159ac0-9481-4406-beaa-e237932a7fb8 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:27:52.562130 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-e5159ac0-9481-4406-beaa-e237932a7fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:27:52.599240 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d8779-4ab2-4c99-926f-9e0cdc2c9932 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:52.639157 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:52.648690 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 62/126] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:27:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:27:52.649956 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5159ac0-9481-4406-beaa-e237932a7fb8 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:52.714028 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b56f721-d8e6-467a-a15f-57dee43679a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:52.714936 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56f721-d8e6-467a-a15f-57dee43679a1 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:52.715223 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56f721-d8e6-467a-a15f-57dee43679a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:52.715505 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56f721-d8e6-467a-a15f-57dee43679a1 None admin] PATH_INFO: `/v3/users/a24c896dd82441dbb4ac8f0fab365a8f` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:52.717665 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b56f721-d8e6-467a-a15f-57dee43679a1 None admin] RBAC: Authorizing `identity:get_user(user_id=a24c896dd82441dbb4ac8f0fab365a8f)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:52.719164 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b56f721-d8e6-467a-a15f-57dee43679a1 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:52.720856 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 65/127] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:27:52 2022] GET /identity/v3/users/a24c896dd82441dbb4ac8f0fab365a8f => generated 275 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:52.721711 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b56f721-d8e6-467a-a15f-57dee43679a1 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:52.787348 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5159ac0-9481-4406-beaa-e237932a7fb8 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:52.839554 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c600dd04-84f2-498c-a377-102664bbeb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:52.840495 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600dd04-84f2-498c-a377-102664bbeb13 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:52.841205 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600dd04-84f2-498c-a377-102664bbeb13 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:52.841503 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600dd04-84f2-498c-a377-102664bbeb13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:52.864816 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b56f721-d8e6-467a-a15f-57dee43679a1 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:52.915286 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb370edf-7772-40a3-a3a5-986ef2185fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:52.916546 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb370edf-7772-40a3-a3a5-986ef2185fc1 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:52.916842 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb370edf-7772-40a3-a3a5-986ef2185fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:52.917129 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb370edf-7772-40a3-a3a5-986ef2185fc1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:52.928539 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:52.938526 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb370edf-7772-40a3-a3a5-986ef2185fc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:27:52.943234 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c600dd04-84f2-498c-a377-102664bbeb13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:52.944835 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c600dd04-84f2-498c-a377-102664bbeb13 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:52.948817 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-eb370edf-7772-40a3-a3a5-986ef2185fc1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 16:27:52.948817 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:27:52.948817 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:27:52.948817 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:27:52.948817 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:27:52.948817 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:27:52.948817 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:27:52.948817 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:27:52.948817 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:27:52.948817 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:27:52.948817 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:27:52.948817 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:27:52.948817 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:27:52.948817 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:27:52.948817 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:27:52.948817 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:27:52.949708 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:27:52.949708 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:27:52.949708 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:27:52.949708 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:27:52.949708 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:27:52.949708 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:27:52.949708 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:27:52.949708 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:27:52.949708 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:27:52.949708 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:27:52.949708 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:27:52.949708 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:27:52.949708 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:27:52.949708 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:27:52.949708 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:27:52.949708 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:27:52.950471 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:27:52.950471 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:27:52.950471 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:27:52.950471 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:27:52.950471 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:27:52.950471 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:27:52.950471 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:27:52.950471 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:27:52.950471 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:27:52.950471 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:27:52.950471 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 16:27:52.950471 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:27:52.950471 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 66/128] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:27:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:27:52.951271 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb370edf-7772-40a3-a3a5-986ef2185fc1 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:52.966007 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:53.083954 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 63/129] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:27:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 16:27:53.084740 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c600dd04-84f2-498c-a377-102664bbeb13 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:53.150600 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb370edf-7772-40a3-a3a5-986ef2185fc1 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:53.230919 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:53.242381 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c600dd04-84f2-498c-a377-102664bbeb13 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:53.245764 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76df72b-9b5e-49e4-8956-3d85df1f6bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:53.247136 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76df72b-9b5e-49e4-8956-3d85df1f6bc4 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:53.247612 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76df72b-9b5e-49e4-8956-3d85df1f6bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:53.248081 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76df72b-9b5e-49e4-8956-3d85df1f6bc4 None admin] PATH_INFO: `/v3/projects/d4a456cd7f83414a895ab7982fec2a68` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:53.258367 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76df72b-9b5e-49e4-8956-3d85df1f6bc4 None admin] RBAC: Authorizing `identity:get_project(project_id=d4a456cd7f83414a895ab7982fec2a68)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:53.260158 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76df72b-9b5e-49e4-8956-3d85df1f6bc4 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:53.262216 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 67/130] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:27:52 2022] GET /identity/v3/projects/d4a456cd7f83414a895ab7982fec2a68 => generated 298 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:53.262865 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76df72b-9b5e-49e4-8956-3d85df1f6bc4 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:53.280058 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aff8a8-d7fe-4d04-a5f9-9bf7cd13cf26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:53.280776 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aff8a8-d7fe-4d04-a5f9-9bf7cd13cf26 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:53.281072 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aff8a8-d7fe-4d04-a5f9-9bf7cd13cf26 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:53.281415 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aff8a8-d7fe-4d04-a5f9-9bf7cd13cf26 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:53.282480 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aff8a8-d7fe-4d04-a5f9-9bf7cd13cf26 None admin] RBAC: Adding query filter params (name=member) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:27:53.282896 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aff8a8-d7fe-4d04-a5f9-9bf7cd13cf26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:53.285307 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aff8a8-d7fe-4d04-a5f9-9bf7cd13cf26 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:53.297547 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 64/131] 10.222.0.36 () {56 vars in 1101 bytes} [Thu May 19 16:27:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:53.298664 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aff8a8-d7fe-4d04-a5f9-9bf7cd13cf26 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:53.429530 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:53.473981 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aff8a8-d7fe-4d04-a5f9-9bf7cd13cf26 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:53.488096 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76df72b-9b5e-49e4-8956-3d85df1f6bc4 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:53.540563 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8ed83d-176a-4822-b5e7-552fefe643e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:53.542411 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8ed83d-176a-4822-b5e7-552fefe643e0 None admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:53.543050 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8ed83d-176a-4822-b5e7-552fefe643e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:53.543710 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8ed83d-176a-4822-b5e7-552fefe643e0 None admin] PATH_INFO: `/v3/projects/d4a456cd7f83414a895ab7982fec2a68/users/a24c896dd82441dbb4ac8f0fab365a8f/roles/50f62a33eb434682baa7529619fe14b0` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:53.548068 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8ed83d-176a-4822-b5e7-552fefe643e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=d4a456cd7f83414a895ab7982fec2a68, user_id=a24c896dd82441dbb4ac8f0fab365a8f, role_id=50f62a33eb434682baa7529619fe14b0)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:53.551100 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8ed83d-176a-4822-b5e7-552fefe643e0 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:53.568451 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0605b61-285e-4cc2-8fbe-65c46462d5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:53.569696 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0605b61-285e-4cc2-8fbe-65c46462d5d7 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:53.570127 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0605b61-285e-4cc2-8fbe-65c46462d5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:53.570546 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0605b61-285e-4cc2-8fbe-65c46462d5d7 None admin] PATH_INFO: `/v3/projects/d4a456cd7f83414a895ab7982fec2a68` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:53.575678 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 65/132] 10.222.0.36 () {58 vars in 1439 bytes} [Thu May 19 16:27:53 2022] PUT /identity/v3/projects/d4a456cd7f83414a895ab7982fec2a68/users/a24c896dd82441dbb4ac8f0fab365a8f/roles/50f62a33eb434682baa7529619fe14b0 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:27:53.576559 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0605b61-285e-4cc2-8fbe-65c46462d5d7 None admin] RBAC: Authorizing `identity:get_project(project_id=d4a456cd7f83414a895ab7982fec2a68)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:53.577474 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8ed83d-176a-4822-b5e7-552fefe643e0 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:53.578393 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0605b61-285e-4cc2-8fbe-65c46462d5d7 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:53.580300 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 68/133] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:27:53 2022] GET /identity/v3/projects/d4a456cd7f83414a895ab7982fec2a68 => generated 298 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:53.587391 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0605b61-285e-4cc2-8fbe-65c46462d5d7 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:53.613505 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:53.809680 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8ed83d-176a-4822-b5e7-552fefe643e0 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:53.833749 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0605b61-285e-4cc2-8fbe-65c46462d5d7 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:53.880365 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7100a744-eaf6-4742-989a-0b18a65f7422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:53.882760 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100a744-eaf6-4742-989a-0b18a65f7422 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:53.883395 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100a744-eaf6-4742-989a-0b18a65f7422 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:53.884053 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100a744-eaf6-4742-989a-0b18a65f7422 None admin] PATH_INFO: `/v3/users/a24c896dd82441dbb4ac8f0fab365a8f` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:53.887872 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7100a744-eaf6-4742-989a-0b18a65f7422 None admin] RBAC: Authorizing `identity:get_user(user_id=a24c896dd82441dbb4ac8f0fab365a8f)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:53.891207 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7100a744-eaf6-4742-989a-0b18a65f7422 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:53.896279 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 66/134] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:27:53 2022] GET /identity/v3/users/a24c896dd82441dbb4ac8f0fab365a8f => generated 275 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:53.905049 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7100a744-eaf6-4742-989a-0b18a65f7422 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:53.919516 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ca333c-4293-46d8-b4dc-fd6c6ba63d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:53.920193 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca333c-4293-46d8-b4dc-fd6c6ba63d50 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:53.920487 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca333c-4293-46d8-b4dc-fd6c6ba63d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:53.920770 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca333c-4293-46d8-b4dc-fd6c6ba63d50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:53.921662 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ca333c-4293-46d8-b4dc-fd6c6ba63d50 None admin] RBAC: Adding query filter params (role.id=156d234cfe9d4844933886f1ba568694, scope.project.id=d4a456cd7f83414a895ab7982fec2a68, user.id=16bccb2c02f64fb0892b8adc61acbd33) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:27:53.924392 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ca333c-4293-46d8-b4dc-fd6c6ba63d50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:53.924392 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ca333c-4293-46d8-b4dc-fd6c6ba63d50 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:53.946195 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 69/135] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:27:53 2022] GET /identity/v3/role_assignments?user.id=16bccb2c02f64fb0892b8adc61acbd33&scope.project.id=d4a456cd7f83414a895ab7982fec2a68&role.id=156d234cfe9d4844933886f1ba568694 => generated 259 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:27:54.050729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:54.071195 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7100a744-eaf6-4742-989a-0b18a65f7422 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:54.117019 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbc3b9f-2058-4280-b9ed-f9fa8b899ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:54.117929 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc3b9f-2058-4280-b9ed-f9fa8b899ced None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:54.118219 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc3b9f-2058-4280-b9ed-f9fa8b899ced None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:54.118502 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc3b9f-2058-4280-b9ed-f9fa8b899ced None admin] PATH_INFO: `/v3/projects/e2a683cd4fca4fe5ab8a9837a71d5988` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:54.119718 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbc3b9f-2058-4280-b9ed-f9fa8b899ced None admin] RBAC: Authorizing `identity:get_project(project_id=e2a683cd4fca4fe5ab8a9837a71d5988)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:54.121936 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede6a99-499a-4409-a957-88fb4b03bd7b None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:54.122234 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede6a99-499a-4409-a957-88fb4b03bd7b None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:54.122514 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede6a99-499a-4409-a957-88fb4b03bd7b None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:54.123973 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 70/136] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:27:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 16:27:54.125060 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbc3b9f-2058-4280-b9ed-f9fa8b899ced None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:54.126920 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 67/137] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:27:53 2022] GET /identity/v3/projects/e2a683cd4fca4fe5ab8a9837a71d5988 => generated 312 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:54.133093 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede6a99-499a-4409-a957-88fb4b03bd7b None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:54.135436 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4becf315-c434-4689-b5e4-000b9cd75ad7 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:54.135730 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4becf315-c434-4689-b5e4-000b9cd75ad7 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:54.136014 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4becf315-c434-4689-b5e4-000b9cd75ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:54.145540 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:54.172306 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-4becf315-c434-4689-b5e4-000b9cd75ad7 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:27:54.173645 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-4becf315-c434-4689-b5e4-000b9cd75ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:27:54.339577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 68/138] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:27:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:27:54.351565 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede6a99-499a-4409-a957-88fb4b03bd7b None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:54.382991 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4becf315-c434-4689-b5e4-000b9cd75ad7 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:54.440741 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e152a8b-ba06-4c38-8e2c-7eb00972e929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:54.442084 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e152a8b-ba06-4c38-8e2c-7eb00972e929 None admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:54.442572 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e152a8b-ba06-4c38-8e2c-7eb00972e929 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:54.443047 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e152a8b-ba06-4c38-8e2c-7eb00972e929 None admin] PATH_INFO: `/v3/projects/e2a683cd4fca4fe5ab8a9837a71d5988/users/a24c896dd82441dbb4ac8f0fab365a8f/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:54.445359 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e152a8b-ba06-4c38-8e2c-7eb00972e929 None admin] RBAC: Authorizing `identity:create_grant(project_id=e2a683cd4fca4fe5ab8a9837a71d5988, user_id=a24c896dd82441dbb4ac8f0fab365a8f, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:54.447073 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e152a8b-ba06-4c38-8e2c-7eb00972e929 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:54.477696 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 71/139] 10.222.0.36 () {58 vars in 1439 bytes} [Thu May 19 16:27:54 2022] PUT /identity/v3/projects/e2a683cd4fca4fe5ab8a9837a71d5988/users/a24c896dd82441dbb4ac8f0fab365a8f/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:27:54.574849 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:54.601635 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4becf315-c434-4689-b5e4-000b9cd75ad7 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:54.649981 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18abd52b-c463-43e8-b43f-f1911d5bfcd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:54.650838 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18abd52b-c463-43e8-b43f-f1911d5bfcd4 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:54.651128 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18abd52b-c463-43e8-b43f-f1911d5bfcd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:54.651410 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18abd52b-c463-43e8-b43f-f1911d5bfcd4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:54.665803 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18abd52b-c463-43e8-b43f-f1911d5bfcd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:27:54.673656 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-18abd52b-c463-43e8-b43f-f1911d5bfcd4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 16:27:54.673656 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:27:54.673656 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:27:54.673656 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:27:54.673656 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:27:54.673656 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:27:54.673656 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:27:54.673656 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:27:54.673656 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:27:54.673656 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:27:54.673656 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:27:54.673656 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:27:54.673656 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:27:54.673656 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:27:54.673656 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:27:54.673656 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:27:54.674520 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:27:54.674520 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:27:54.674520 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:27:54.674520 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:27:54.674520 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:27:54.674520 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:27:54.674520 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:27:54.674520 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:27:54.674520 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:27:54.674520 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:27:54.674520 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:27:54.674520 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:27:54.674520 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:27:54.674520 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:27:54.674520 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:27:54.674520 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:27:54.675424 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:27:54.675424 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:27:54.675424 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:27:54.675424 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:27:54.675424 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:27:54.675424 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:27:54.675424 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:27:54.675424 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:27:54.675424 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:27:54.675424 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:27:54.675424 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 16:27:54.675424 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:27:54.675424 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 69/140] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:27:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:27:54.681302 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e152a8b-ba06-4c38-8e2c-7eb00972e929 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:54.787487 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:54.830968 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e152a8b-ba06-4c38-8e2c-7eb00972e929 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:54.887270 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ae688-7463-4e32-bc7e-ba02ad6d2931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:54.888362 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ae688-7463-4e32-bc7e-ba02ad6d2931 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:54.888871 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ae688-7463-4e32-bc7e-ba02ad6d2931 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:54.889390 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ae688-7463-4e32-bc7e-ba02ad6d2931 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:54.890406 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2ae688-7463-4e32-bc7e-ba02ad6d2931 None admin] RBAC: Adding query filter params (name=member) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:27:54.890950 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2ae688-7463-4e32-bc7e-ba02ad6d2931 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:54.892763 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2ae688-7463-4e32-bc7e-ba02ad6d2931 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:54.907138 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 72/141] 10.222.0.36 () {56 vars in 1101 bytes} [Thu May 19 16:27:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:54.914818 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18abd52b-c463-43e8-b43f-f1911d5bfcd4 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:55.073919 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18abd52b-c463-43e8-b43f-f1911d5bfcd4 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:55.122191 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c058bf-44e7-4074-b65a-960d2a23fca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:55.123086 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c058bf-44e7-4074-b65a-960d2a23fca6 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:55.123377 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c058bf-44e7-4074-b65a-960d2a23fca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:55.123660 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c058bf-44e7-4074-b65a-960d2a23fca6 None admin] PATH_INFO: `/v3/users/a24c896dd82441dbb4ac8f0fab365a8f` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:55.127016 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c058bf-44e7-4074-b65a-960d2a23fca6 None admin] RBAC: Authorizing `identity:get_user(user_id=a24c896dd82441dbb4ac8f0fab365a8f)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:55.129058 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c058bf-44e7-4074-b65a-960d2a23fca6 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:55.131117 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 70/142] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:27:54 2022] GET /identity/v3/users/a24c896dd82441dbb4ac8f0fab365a8f => generated 275 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:55.136723 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ae688-7463-4e32-bc7e-ba02ad6d2931 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:55.280307 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ae688-7463-4e32-bc7e-ba02ad6d2931 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:55.311847 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:55.321043 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d169c4-42ea-4cf6-872e-17210e6a44bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:55.321751 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d169c4-42ea-4cf6-872e-17210e6a44bf None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:55.322041 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d169c4-42ea-4cf6-872e-17210e6a44bf None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:55.322322 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d169c4-42ea-4cf6-872e-17210e6a44bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:55.340862 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef02cf39-db9f-4d7a-9fe0-bf8fd8ed2357 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:55.341119 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef02cf39-db9f-4d7a-9fe0-bf8fd8ed2357 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:55.341629 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef02cf39-db9f-4d7a-9fe0-bf8fd8ed2357 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:55.343298 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 71/143] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:27:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 16:27:55.354021 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db73220a-6eac-4365-9ad4-b8b8872baf29 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:55.354608 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db73220a-6eac-4365-9ad4-b8b8872baf29 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:55.355101 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db73220a-6eac-4365-9ad4-b8b8872baf29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:55.384565 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:55.391743 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-db73220a-6eac-4365-9ad4-b8b8872baf29 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:27:55.393321 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-db73220a-6eac-4365-9ad4-b8b8872baf29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:27:55.460882 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d169c4-42ea-4cf6-872e-17210e6a44bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:55.462450 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d169c4-42ea-4cf6-872e-17210e6a44bf None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:55.499437 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 72/144] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:27:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 19 16:27:55.535188 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db73220a-6eac-4365-9ad4-b8b8872baf29 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:55.660480 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 73/145] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:27:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 16:27:55.667532 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d169c4-42ea-4cf6-872e-17210e6a44bf None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:55.736640 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:55.753544 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db73220a-6eac-4365-9ad4-b8b8872baf29 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:55.833746 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ac4bc9-3b6a-42f8-b08c-b664ac8645f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:55.835243 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ac4bc9-3b6a-42f8-b08c-b664ac8645f6 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:55.836027 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ac4bc9-3b6a-42f8-b08c-b664ac8645f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:55.836518 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ac4bc9-3b6a-42f8-b08c-b664ac8645f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:55.840542 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:55.860529 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ac4bc9-3b6a-42f8-b08c-b664ac8645f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:27:55.873757 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d169c4-42ea-4cf6-872e-17210e6a44bf None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:55.878459 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-54ac4bc9-3b6a-42f8-b08c-b664ac8645f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:27:55.878459 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:27:55.878459 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:27:55.878459 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:27:55.878459 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:27:55.878459 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:27:55.878459 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:27:55.878459 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:27:55.878459 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:27:55.878459 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:27:55.878459 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:27:55.878459 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:27:55.878459 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:27:55.878459 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:27:55.878459 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:27:55.878459 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:27:55.879305 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:27:55.879305 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:27:55.879305 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:27:55.879305 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:27:55.879305 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:27:55.879305 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:27:55.879305 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:27:55.879305 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:27:55.879305 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:27:55.879305 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:27:55.879305 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:27:55.879305 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:27:55.879305 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:27:55.879305 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:27:55.879305 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:27:55.879305 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:27:55.880083 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:27:55.880083 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:27:55.880083 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:27:55.880083 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:27:55.880083 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:27:55.880083 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:27:55.880083 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:27:55.880083 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:27:55.880083 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:27:55.880083 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:27:55.880083 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:27:55.880083 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:27:55.888439 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 73/146] 10.222.0.36 () {56 vars in 1096 bytes} [Thu May 19 16:27:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 16:27:55.890343 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f98469b-7c28-4504-9126-c8cfdb350300 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:55.890876 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f98469b-7c28-4504-9126-c8cfdb350300 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:55.893567 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f98469b-7c28-4504-9126-c8cfdb350300 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:55.897404 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 74/147] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:27:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:27:55.897404 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f98469b-7c28-4504-9126-c8cfdb350300 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:55.950558 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62d0fb9-8438-49e8-9474-72d06c3a37d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:55.951443 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d0fb9-8438-49e8-9474-72d06c3a37d4 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:55.951729 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d0fb9-8438-49e8-9474-72d06c3a37d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:55.952006 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d0fb9-8438-49e8-9474-72d06c3a37d4 None admin] PATH_INFO: `/v3/projects/d4a456cd7f83414a895ab7982fec2a68` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:55.953333 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62d0fb9-8438-49e8-9474-72d06c3a37d4 None admin] RBAC: Authorizing `identity:get_project(project_id=d4a456cd7f83414a895ab7982fec2a68)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:55.955728 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62d0fb9-8438-49e8-9474-72d06c3a37d4 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:55.965253 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 74/148] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:27:55 2022] GET /identity/v3/projects/d4a456cd7f83414a895ab7982fec2a68 => generated 298 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:55.967643 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa3be2-0342-4f51-8c58-85d5ce79c09a None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:55.968494 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa3be2-0342-4f51-8c58-85d5ce79c09a None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:55.968784 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa3be2-0342-4f51-8c58-85d5ce79c09a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:56.002473 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-c4aa3be2-0342-4f51-8c58-85d5ce79c09a None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:27:56.003720 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-c4aa3be2-0342-4f51-8c58-85d5ce79c09a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:27:56.102489 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f98469b-7c28-4504-9126-c8cfdb350300 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:56.130538 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 75/149] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:27:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:27:56.131816 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aa3be2-0342-4f51-8c58-85d5ce79c09a None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:56.160279 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a889f-b9a8-40c7-8a01-48c4f1038a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:56.160970 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a889f-b9a8-40c7-8a01-48c4f1038a2d None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:56.161286 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a889f-b9a8-40c7-8a01-48c4f1038a2d None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:56.161579 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a889f-b9a8-40c7-8a01-48c4f1038a2d None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:56.162409 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1a889f-b9a8-40c7-8a01-48c4f1038a2d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:27:56.162741 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1a889f-b9a8-40c7-8a01-48c4f1038a2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:56.164157 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1a889f-b9a8-40c7-8a01-48c4f1038a2d None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:56.180157 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:56.186276 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 75/150] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:27:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:56.187140 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a889f-b9a8-40c7-8a01-48c4f1038a2d None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:56.302700 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:56.313610 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aa3be2-0342-4f51-8c58-85d5ce79c09a None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:56.352215 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701c2241-3255-4b6e-bc9c-ed9a4043d042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:56.352996 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c2241-3255-4b6e-bc9c-ed9a4043d042 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:56.353343 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c2241-3255-4b6e-bc9c-ed9a4043d042 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:56.353630 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c2241-3255-4b6e-bc9c-ed9a4043d042 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:56.354607 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701c2241-3255-4b6e-bc9c-ed9a4043d042 None admin] RBAC: Adding query filter params (role.id=0e98fd94a6d24e46b9fe09a22a7df04a, scope.project.id=d4a456cd7f83414a895ab7982fec2a68, user.id=a24c896dd82441dbb4ac8f0fab365a8f) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:27:56.354927 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701c2241-3255-4b6e-bc9c-ed9a4043d042 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:56.356592 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701c2241-3255-4b6e-bc9c-ed9a4043d042 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:56.360012 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a889f-b9a8-40c7-8a01-48c4f1038a2d None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:56.363593 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 76/151] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:27:56 2022] GET /identity/v3/role_assignments?user.id=a24c896dd82441dbb4ac8f0fab365a8f&scope.project.id=d4a456cd7f83414a895ab7982fec2a68&role.id=0e98fd94a6d24e46b9fe09a22a7df04a => generated 608 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:56.372087 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701c2241-3255-4b6e-bc9c-ed9a4043d042 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:56.399435 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877efa4e-5c06-40ed-901a-9271055963ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:56.400378 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877efa4e-5c06-40ed-901a-9271055963ff None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:56.400672 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877efa4e-5c06-40ed-901a-9271055963ff None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:56.400951 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877efa4e-5c06-40ed-901a-9271055963ff None admin] PATH_INFO: `/v3/domains/default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:56.402308 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877efa4e-5c06-40ed-901a-9271055963ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:56.403866 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877efa4e-5c06-40ed-901a-9271055963ff None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:56.405759 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 76/152] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:27:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:56.414269 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877efa4e-5c06-40ed-901a-9271055963ff None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:56.505534 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:56.522161 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701c2241-3255-4b6e-bc9c-ed9a4043d042 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:56.533707 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877efa4e-5c06-40ed-901a-9271055963ff None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:56.564208 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:56.579090 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326bfa99-6a63-4c75-a01c-f04f152ee76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:56.579796 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326bfa99-6a63-4c75-a01c-f04f152ee76c None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:56.580085 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326bfa99-6a63-4c75-a01c-f04f152ee76c None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:56.580369 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326bfa99-6a63-4c75-a01c-f04f152ee76c None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:56.589797 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326bfa99-6a63-4c75-a01c-f04f152ee76c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:56.591682 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326bfa99-6a63-4c75-a01c-f04f152ee76c None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:56.594712 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ed3e06-5750-44a1-8360-baf0c8e28a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:56.595625 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed3e06-5750-44a1-8360-baf0c8e28a08 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:56.595955 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed3e06-5750-44a1-8360-baf0c8e28a08 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:56.596240 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed3e06-5750-44a1-8360-baf0c8e28a08 None admin] PATH_INFO: `/v3/users/ac4bfc4d6198473c8d280a14b2c6036f` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:56.599447 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ed3e06-5750-44a1-8360-baf0c8e28a08 None admin] RBAC: Authorizing `identity:get_user(user_id=ac4bfc4d6198473c8d280a14b2c6036f)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:56.601048 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ed3e06-5750-44a1-8360-baf0c8e28a08 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:56.603210 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 77/153] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:27:56 2022] GET /identity/v3/users/ac4bfc4d6198473c8d280a14b2c6036f => generated 247 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:56.610855 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ed3e06-5750-44a1-8360-baf0c8e28a08 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:56.620978 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 77/154] 10.222.0.36 () {60 vars in 1132 bytes} [Thu May 19 16:27:56 2022] POST /identity/v3/users => generated 283 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:27:56.795059 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ed3e06-5750-44a1-8360-baf0c8e28a08 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:56.843405 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba924c-62a9-40ce-a640-f1aa9d5b328a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:56.844470 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba924c-62a9-40ce-a640-f1aa9d5b328a None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:56.844971 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba924c-62a9-40ce-a640-f1aa9d5b328a None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:56.845461 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba924c-62a9-40ce-a640-f1aa9d5b328a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:56.884539 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:56.958756 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eba924c-62a9-40ce-a640-f1aa9d5b328a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:56.960654 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eba924c-62a9-40ce-a640-f1aa9d5b328a None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:57.079266 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 78/155] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:27:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 16:27:57.086627 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326bfa99-6a63-4c75-a01c-f04f152ee76c None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:57.139693 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb17a6-a98c-4da2-b3c8-6047c328a096 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:57.139981 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb17a6-a98c-4da2-b3c8-6047c328a096 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:57.140166 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb17a6-a98c-4da2-b3c8-6047c328a096 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:57.141919 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 79/156] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:27:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:27:57.148262 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc9539-101b-4d52-a6d5-26bd6db07fe7 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:57.149012 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc9539-101b-4d52-a6d5-26bd6db07fe7 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:57.149367 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc9539-101b-4d52-a6d5-26bd6db07fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:57.170636 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-7fdc9539-101b-4d52-a6d5-26bd6db07fe7 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:27:57.172202 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-7fdc9539-101b-4d52-a6d5-26bd6db07fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:27:57.223237 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326bfa99-6a63-4c75-a01c-f04f152ee76c None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:57.252478 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:57.267844 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22db2965-b5d0-40ad-884b-ac3757f64ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:57.269748 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db2965-b5d0-40ad-884b-ac3757f64ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:57.270272 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db2965-b5d0-40ad-884b-ac3757f64ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:57.270754 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db2965-b5d0-40ad-884b-ac3757f64ec6 None admin] PATH_INFO: `/v3/projects/d4a456cd7f83414a895ab7982fec2a68` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:57.272647 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22db2965-b5d0-40ad-884b-ac3757f64ec6 None admin] RBAC: Authorizing `identity:get_project(project_id=d4a456cd7f83414a895ab7982fec2a68)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:57.274645 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22db2965-b5d0-40ad-884b-ac3757f64ec6 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:57.278708 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 80/157] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:27:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:27:57.278839 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 78/158] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:27:57 2022] GET /identity/v3/projects/d4a456cd7f83414a895ab7982fec2a68 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:57.284322 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdc9539-101b-4d52-a6d5-26bd6db07fe7 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:57.303100 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22db2965-b5d0-40ad-884b-ac3757f64ec6 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:57.371600 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:57.410747 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdc9539-101b-4d52-a6d5-26bd6db07fe7 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:57.442752 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22db2965-b5d0-40ad-884b-ac3757f64ec6 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:57.457517 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff5a10b-db2a-4b8e-97e7-fe6648074586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:57.458514 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5a10b-db2a-4b8e-97e7-fe6648074586 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:57.458983 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5a10b-db2a-4b8e-97e7-fe6648074586 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:57.459440 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5a10b-db2a-4b8e-97e7-fe6648074586 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:57.460489 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff5a10b-db2a-4b8e-97e7-fe6648074586 None admin] RBAC: Adding query filter params (role.id=e2915fe109244a07a9be96bd2b219fdd, scope.project.id=d4a456cd7f83414a895ab7982fec2a68, user.id=ac4bfc4d6198473c8d280a14b2c6036f) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:27:57.461002 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff5a10b-db2a-4b8e-97e7-fe6648074586 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:57.463453 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff5a10b-db2a-4b8e-97e7-fe6648074586 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:57.471181 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 81/159] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:27:57 2022] GET /identity/v3/role_assignments?user.id=ac4bfc4d6198473c8d280a14b2c6036f&scope.project.id=d4a456cd7f83414a895ab7982fec2a68&role.id=e2915fe109244a07a9be96bd2b219fdd => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:57.472892 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd755d99-0105-4c04-ac15-693b94d2b63e None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:57.473423 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd755d99-0105-4c04-ac15-693b94d2b63e None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:57.473926 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd755d99-0105-4c04-ac15-693b94d2b63e None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:57.475564 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 82/160] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:27:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:27:57.486801 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1c2aab-1163-46c9-a25e-bad0d55fd3e8 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:57.487479 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1c2aab-1163-46c9-a25e-bad0d55fd3e8 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:57.487967 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1c2aab-1163-46c9-a25e-bad0d55fd3e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:57.507299 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c1b4b-d001-4e41-90a2-3e1d96574595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:57.508579 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c1b4b-d001-4e41-90a2-3e1d96574595 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:57.509207 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c1b4b-d001-4e41-90a2-3e1d96574595 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:57.512841 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c1b4b-d001-4e41-90a2-3e1d96574595 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:57.516410 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-9e1c2aab-1163-46c9-a25e-bad0d55fd3e8 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:27:57.519659 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-9e1c2aab-1163-46c9-a25e-bad0d55fd3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:27:57.538989 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383c1b4b-d001-4e41-90a2-3e1d96574595 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:27:57.552857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-383c1b4b-d001-4e41-90a2-3e1d96574595 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 16:27:57.552857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:27:57.552857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:27:57.552857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:27:57.552857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:27:57.552857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:27:57.552857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:27:57.552857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:27:57.552857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:27:57.552857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:27:57.552857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:27:57.552857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:27:57.552857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:27:57.552857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:27:57.552857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:27:57.552857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:27:57.553831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:27:57.553831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:27:57.553831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:27:57.553831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:27:57.553831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:27:57.553831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:27:57.553831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:27:57.553831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:27:57.553831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:27:57.553831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:27:57.553831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:27:57.553831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:27:57.553831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:27:57.553831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:27:57.553831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:27:57.553831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:27:57.554616 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:27:57.554616 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:27:57.554616 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:27:57.554616 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:27:57.554616 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:27:57.554616 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:27:57.554616 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:27:57.554616 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:27:57.554616 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:27:57.554616 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:27:57.554616 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 16:27:57.554616 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:27:57.558110 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 79/161] 10.222.0.36 () {56 vars in 1114 bytes} [Thu May 19 16:27:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:27:57.564753 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c1b4b-d001-4e41-90a2-3e1d96574595 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:57.646118 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:57.678838 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 83/162] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:27:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:27:57.704252 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1c2aab-1163-46c9-a25e-bad0d55fd3e8 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:57.726718 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c1b4b-d001-4e41-90a2-3e1d96574595 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:57.742091 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:57.791203 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34db74b5-bc1b-4354-9103-3fa15dc92652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:57.792279 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34db74b5-bc1b-4354-9103-3fa15dc92652 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:57.792806 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34db74b5-bc1b-4354-9103-3fa15dc92652 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:57.795083 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34db74b5-bc1b-4354-9103-3fa15dc92652 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:57.796506 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34db74b5-bc1b-4354-9103-3fa15dc92652 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:27:57.797679 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34db74b5-bc1b-4354-9103-3fa15dc92652 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:57.800091 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34db74b5-bc1b-4354-9103-3fa15dc92652 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:57.819131 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 80/163] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:27:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:27:57.846765 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34db74b5-bc1b-4354-9103-3fa15dc92652 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:57.903396 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1c2aab-1163-46c9-a25e-bad0d55fd3e8 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:57.961070 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d36d936-1f80-460b-af1f-b25c34639afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:57.962306 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36d936-1f80-460b-af1f-b25c34639afa None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:57.962834 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36d936-1f80-460b-af1f-b25c34639afa None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:57.963344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36d936-1f80-460b-af1f-b25c34639afa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:57.979000 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d36d936-1f80-460b-af1f-b25c34639afa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:27:57.991750 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-3d36d936-1f80-460b-af1f-b25c34639afa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 16:27:57.991750 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:27:57.991750 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:27:57.991750 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:27:57.991750 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:27:57.991750 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:27:57.991750 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:27:57.991750 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:27:57.991750 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:27:57.991750 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:27:57.991750 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:27:57.991750 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:27:57.991750 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:27:57.991750 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:27:57.991750 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:27:57.991750 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:27:57.992721 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:27:57.992721 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:27:57.992721 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:27:57.992721 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:27:57.992721 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:27:57.992721 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:27:57.992721 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:27:57.992721 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:27:57.992721 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:27:57.992721 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:27:57.992721 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:27:57.992721 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:27:57.992721 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:27:57.992721 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:27:57.992721 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:27:57.992721 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:27:57.993536 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:27:57.993536 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:27:57.993536 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:27:57.993536 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:27:57.993536 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:27:57.993536 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:27:57.993536 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:27:57.993536 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:27:57.993536 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:27:57.993536 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:27:57.993536 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 16:27:57.993536 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:27:57.996700 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 84/164] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:27:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:27:57.999609 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e322e-622d-4028-aaa8-f74709ef488a None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:57.999789 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34db74b5-bc1b-4354-9103-3fa15dc92652 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:58.000311 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e322e-622d-4028-aaa8-f74709ef488a None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:58.000795 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e322e-622d-4028-aaa8-f74709ef488a None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:58.006486 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 85/165] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:27:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:27:58.007662 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340e322e-622d-4028-aaa8-f74709ef488a None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:58.055310 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d877d6-845f-43d6-836f-975249bac434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:58.056216 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d877d6-845f-43d6-836f-975249bac434 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:58.056510 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d877d6-845f-43d6-836f-975249bac434 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:58.056898 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d877d6-845f-43d6-836f-975249bac434 None admin] PATH_INFO: `/v3/users/a24c896dd82441dbb4ac8f0fab365a8f` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:58.061978 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d877d6-845f-43d6-836f-975249bac434 None admin] RBAC: Authorizing `identity:get_user(user_id=a24c896dd82441dbb4ac8f0fab365a8f)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:58.063488 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d877d6-845f-43d6-836f-975249bac434 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:58.070410 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 81/166] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:27:57 2022] GET /identity/v3/users/a24c896dd82441dbb4ac8f0fab365a8f => generated 275 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:58.070410 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300a1f5-cebe-4e76-93cd-0531429263f7 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:58.070410 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300a1f5-cebe-4e76-93cd-0531429263f7 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:58.070410 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300a1f5-cebe-4e76-93cd-0531429263f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:58.082173 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:58.103357 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-0300a1f5-cebe-4e76-93cd-0531429263f7 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:27:58.104576 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-0300a1f5-cebe-4e76-93cd-0531429263f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:27:58.139910 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340e322e-622d-4028-aaa8-f74709ef488a None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:58.159387 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:58.199957 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63b60cf-5c80-41a7-82db-ba0348563a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:58.200896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b60cf-5c80-41a7-82db-ba0348563a4e None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:58.201393 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b60cf-5c80-41a7-82db-ba0348563a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:58.201821 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b60cf-5c80-41a7-82db-ba0348563a4e None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:58.202754 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63b60cf-5c80-41a7-82db-ba0348563a4e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:27:58.204568 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63b60cf-5c80-41a7-82db-ba0348563a4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:58.206233 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63b60cf-5c80-41a7-82db-ba0348563a4e None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:58.222036 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 82/167] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:27:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:27:58.225088 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0300a1f5-cebe-4e76-93cd-0531429263f7 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:58.230613 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 86/168] 10.222.0.36 () {56 vars in 1101 bytes} [Thu May 19 16:27:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:58.231706 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63b60cf-5c80-41a7-82db-ba0348563a4e None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:58.389583 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0300a1f5-cebe-4e76-93cd-0531429263f7 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:58.445320 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235c81a0-18a0-4eed-8547-94bae8fcfee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:58.446185 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235c81a0-18a0-4eed-8547-94bae8fcfee4 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:58.446478 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235c81a0-18a0-4eed-8547-94bae8fcfee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:58.446758 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235c81a0-18a0-4eed-8547-94bae8fcfee4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:58.447503 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63b60cf-5c80-41a7-82db-ba0348563a4e None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:58.461598 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:58.495157 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:58.523011 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe3309-37b7-4dc2-9505-f19ad5f9c545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:58.523940 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe3309-37b7-4dc2-9505-f19ad5f9c545 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:58.524223 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe3309-37b7-4dc2-9505-f19ad5f9c545 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:58.524500 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe3309-37b7-4dc2-9505-f19ad5f9c545 None admin] PATH_INFO: `/v3/users/16bccb2c02f64fb0892b8adc61acbd33` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:58.527368 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe3309-37b7-4dc2-9505-f19ad5f9c545 None admin] RBAC: Authorizing `identity:get_user(user_id=16bccb2c02f64fb0892b8adc61acbd33)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:58.528866 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe3309-37b7-4dc2-9505-f19ad5f9c545 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:58.530912 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 87/169] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:27:58 2022] GET /identity/v3/users/16bccb2c02f64fb0892b8adc61acbd33 => generated 289 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:58.531699 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe3309-37b7-4dc2-9505-f19ad5f9c545 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:58.533736 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235c81a0-18a0-4eed-8547-94bae8fcfee4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:58.535646 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235c81a0-18a0-4eed-8547-94bae8fcfee4 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:58.701529 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe3309-37b7-4dc2-9505-f19ad5f9c545 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:58.738857 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de1da74-b317-4d4c-b70f-4692ac0e1fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:58.739963 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de1da74-b317-4d4c-b70f-4692ac0e1fff None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:58.740496 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de1da74-b317-4d4c-b70f-4692ac0e1fff None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:58.740986 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de1da74-b317-4d4c-b70f-4692ac0e1fff None admin] PATH_INFO: `/v3/roles/member` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:58.742241 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 83/170] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:27:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 16:27:58.743199 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235c81a0-18a0-4eed-8547-94bae8fcfee4 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:58.753495 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de1da74-b317-4d4c-b70f-4692ac0e1fff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:27:58.761972 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-7de1da74-b317-4d4c-b70f-4692ac0e1fff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 16:27:58.761972 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:27:58.761972 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:27:58.761972 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:27:58.761972 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:27:58.761972 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:27:58.761972 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:27:58.761972 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:27:58.761972 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:27:58.761972 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:27:58.761972 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:27:58.761972 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:27:58.761972 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:27:58.761972 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:27:58.761972 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:27:58.761972 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:27:58.762870 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:27:58.762870 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:27:58.762870 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:27:58.762870 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:27:58.762870 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:27:58.762870 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:27:58.762870 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:27:58.762870 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:27:58.762870 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:27:58.762870 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:27:58.762870 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:27:58.762870 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:27:58.762870 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:27:58.762870 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:27:58.762870 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:27:58.762870 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:27:58.763661 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:27:58.763661 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:27:58.763661 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:27:58.763661 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:27:58.763661 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:27:58.763661 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:27:58.763661 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:27:58.763661 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:27:58.763661 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:27:58.763661 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:27:58.763661 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 16:27:58.763661 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:27:58.765936 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 88/171] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:27:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:27:58.766876 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de1da74-b317-4d4c-b70f-4692ac0e1fff None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:58.812705 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:58.900004 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:58.942454 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de1da74-b317-4d4c-b70f-4692ac0e1fff None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:58.962660 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235c81a0-18a0-4eed-8547-94bae8fcfee4 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:59.018278 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2781bc46-e2b2-472e-8bf6-9b8ee3ccaa38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:59.019674 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2781bc46-e2b2-472e-8bf6-9b8ee3ccaa38 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:59.019964 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2781bc46-e2b2-472e-8bf6-9b8ee3ccaa38 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:59.020247 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2781bc46-e2b2-472e-8bf6-9b8ee3ccaa38 None admin] PATH_INFO: `/v3/projects/d4a456cd7f83414a895ab7982fec2a68` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:59.024776 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2781bc46-e2b2-472e-8bf6-9b8ee3ccaa38 None admin] RBAC: Authorizing `identity:get_project(project_id=d4a456cd7f83414a895ab7982fec2a68)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:59.026343 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2781bc46-e2b2-472e-8bf6-9b8ee3ccaa38 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:59.028261 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 89/172] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:27:58 2022] GET /identity/v3/projects/d4a456cd7f83414a895ab7982fec2a68 => generated 298 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:59.035466 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2781bc46-e2b2-472e-8bf6-9b8ee3ccaa38 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:59.054616 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa702e-fefc-4d62-b28b-a2148f1650db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:59.055546 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa702e-fefc-4d62-b28b-a2148f1650db None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:59.055838 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa702e-fefc-4d62-b28b-a2148f1650db None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:59.056121 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa702e-fefc-4d62-b28b-a2148f1650db None admin] PATH_INFO: `/v3/projects/d4a456cd7f83414a895ab7982fec2a68` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:59.057608 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aa702e-fefc-4d62-b28b-a2148f1650db None admin] RBAC: Authorizing `identity:get_project(project_id=d4a456cd7f83414a895ab7982fec2a68)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:59.059279 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aa702e-fefc-4d62-b28b-a2148f1650db None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:59.061319 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 84/173] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:27:58 2022] GET /identity/v3/projects/d4a456cd7f83414a895ab7982fec2a68 => generated 298 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:59.062160 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa702e-fefc-4d62-b28b-a2148f1650db None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:59.215330 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:59.222809 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2781bc46-e2b2-472e-8bf6-9b8ee3ccaa38 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:59.232356 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa702e-fefc-4d62-b28b-a2148f1650db None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:59.272301 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad60a5-5e0a-414e-942a-c81184813381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:59.272949 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad60a5-5e0a-414e-942a-c81184813381 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:59.273276 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad60a5-5e0a-414e-942a-c81184813381 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:59.273569 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad60a5-5e0a-414e-942a-c81184813381 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:59.274435 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ad60a5-5e0a-414e-942a-c81184813381 None admin] RBAC: Adding query filter params (role.id=50f62a33eb434682baa7529619fe14b0, scope.project.id=d4a456cd7f83414a895ab7982fec2a68, user.id=a24c896dd82441dbb4ac8f0fab365a8f) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:27:59.274943 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ad60a5-5e0a-414e-942a-c81184813381 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:59.276417 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ad60a5-5e0a-414e-942a-c81184813381 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:59.283359 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 85/174] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:27:59 2022] GET /identity/v3/role_assignments?user.id=a24c896dd82441dbb4ac8f0fab365a8f&scope.project.id=d4a456cd7f83414a895ab7982fec2a68&role.id=50f62a33eb434682baa7529619fe14b0 => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:59.284779 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad60a5-5e0a-414e-942a-c81184813381 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:59.370008 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e13571-3e1a-4696-9146-f0f14f13f3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:59.370711 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e13571-3e1a-4696-9146-f0f14f13f3b2 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:59.371002 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e13571-3e1a-4696-9146-f0f14f13f3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:59.371367 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:59.371502 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e13571-3e1a-4696-9146-f0f14f13f3b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:59.372043 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e13571-3e1a-4696-9146-f0f14f13f3b2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:27:59.372534 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e13571-3e1a-4696-9146-f0f14f13f3b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:59.374000 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e13571-3e1a-4696-9146-f0f14f13f3b2 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:59.403585 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 90/175] 10.222.0.36 () {56 vars in 1101 bytes} [Thu May 19 16:27:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:59.410251 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e13571-3e1a-4696-9146-f0f14f13f3b2 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:59.422683 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad60a5-5e0a-414e-942a-c81184813381 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:59.465418 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423dc910-f041-4088-8177-80de39bd218a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:59.466959 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423dc910-f041-4088-8177-80de39bd218a None admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:59.467456 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423dc910-f041-4088-8177-80de39bd218a None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:59.467988 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423dc910-f041-4088-8177-80de39bd218a None admin] PATH_INFO: `/v3/projects/d4a456cd7f83414a895ab7982fec2a68/users/16bccb2c02f64fb0892b8adc61acbd33/roles/156d234cfe9d4844933886f1ba568694` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:59.470644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423dc910-f041-4088-8177-80de39bd218a None admin] RBAC: Authorizing `identity:create_grant(project_id=d4a456cd7f83414a895ab7982fec2a68, user_id=16bccb2c02f64fb0892b8adc61acbd33, role_id=156d234cfe9d4844933886f1ba568694)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:59.473001 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423dc910-f041-4088-8177-80de39bd218a None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:59.490793 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 86/176] 10.222.0.36 () {58 vars in 1439 bytes} [Thu May 19 16:27:59 2022] PUT /identity/v3/projects/d4a456cd7f83414a895ab7982fec2a68/users/16bccb2c02f64fb0892b8adc61acbd33/roles/156d234cfe9d4844933886f1ba568694 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:27:59.648733 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e13571-3e1a-4696-9146-f0f14f13f3b2 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:59.707545 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22030d1-4cde-45d3-b7bd-dcdcccf0ce50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:59.708437 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22030d1-4cde-45d3-b7bd-dcdcccf0ce50 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:59.708719 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22030d1-4cde-45d3-b7bd-dcdcccf0ce50 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:59.708993 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22030d1-4cde-45d3-b7bd-dcdcccf0ce50 None admin] PATH_INFO: `/v3/users/a24c896dd82441dbb4ac8f0fab365a8f` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:27:59.714109 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22030d1-4cde-45d3-b7bd-dcdcccf0ce50 None admin] RBAC: Authorizing `identity:get_user(user_id=a24c896dd82441dbb4ac8f0fab365a8f)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:27:59.715709 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22030d1-4cde-45d3-b7bd-dcdcccf0ce50 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:27:59.717860 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 91/177] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:27:59 2022] GET /identity/v3/users/a24c896dd82441dbb4ac8f0fab365a8f => generated 275 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:27:59.723699 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423dc910-f041-4088-8177-80de39bd218a None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:27:59.822419 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:27:59.872330 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423dc910-f041-4088-8177-80de39bd218a None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:27:59.946164 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13926b8f-2d70-46b9-83d6-0b93c57e5856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:27:59.947140 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13926b8f-2d70-46b9-83d6-0b93c57e5856 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:27:59.947430 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13926b8f-2d70-46b9-83d6-0b93c57e5856 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:27:59.947716 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13926b8f-2d70-46b9-83d6-0b93c57e5856 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:00.045536 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13926b8f-2d70-46b9-83d6-0b93c57e5856 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:00.047091 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13926b8f-2d70-46b9-83d6-0b93c57e5856 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:00.145749 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c5bb5-dc4d-46b6-887d-ea09ebf47e50 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:00.146074 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c5bb5-dc4d-46b6-887d-ea09ebf47e50 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:00.146317 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c5bb5-dc4d-46b6-887d-ea09ebf47e50 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:00.154338 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 92/178] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:00 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:00.169205 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b19dcce-2469-4334-a8e4-d5ed6b45ec35 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:00.169205 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b19dcce-2469-4334-a8e4-d5ed6b45ec35 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:00.169205 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b19dcce-2469-4334-a8e4-d5ed6b45ec35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:00.211004 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:00.213565 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-8b19dcce-2469-4334-a8e4-d5ed6b45ec35 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:00.215509 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-8b19dcce-2469-4334-a8e4-d5ed6b45ec35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:00.276716 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:00.309954 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 87/179] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:27:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 583 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 16:28:00.318630 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13926b8f-2d70-46b9-83d6-0b93c57e5856 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:00.384258 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 93/180] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:28:00.408727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023af100-94c7-4522-b517-fa8cc132548b None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:00.409881 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023af100-94c7-4522-b517-fa8cc132548b None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:00.410542 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023af100-94c7-4522-b517-fa8cc132548b None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:00.416524 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 94/181] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:00 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:00.417749 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023af100-94c7-4522-b517-fa8cc132548b None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:00.512362 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13926b8f-2d70-46b9-83d6-0b93c57e5856 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:00.563738 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a91c033-2601-4161-bdca-ccee528dfdde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:00.564854 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a91c033-2601-4161-bdca-ccee528dfdde None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:00.565171 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a91c033-2601-4161-bdca-ccee528dfdde None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:00.565969 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023af100-94c7-4522-b517-fa8cc132548b None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:00.566078 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a91c033-2601-4161-bdca-ccee528dfdde None admin] PATH_INFO: `/v3/projects/e2a683cd4fca4fe5ab8a9837a71d5988` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:00.567337 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a91c033-2601-4161-bdca-ccee528dfdde None admin] RBAC: Authorizing `identity:get_project(project_id=e2a683cd4fca4fe5ab8a9837a71d5988)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:00.569940 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a91c033-2601-4161-bdca-ccee528dfdde None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:00.572021 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 88/182] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:28:00 2022] GET /identity/v3/projects/e2a683cd4fca4fe5ab8a9837a71d5988 => generated 312 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:00.573176 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55ba62-e5b2-4849-9ab1-8ca58936af18 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:00.573462 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55ba62-e5b2-4849-9ab1-8ca58936af18 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:00.573750 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55ba62-e5b2-4849-9ab1-8ca58936af18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:00.598912 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-bf55ba62-e5b2-4849-9ab1-8ca58936af18 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:00.600793 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-bf55ba62-e5b2-4849-9ab1-8ca58936af18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:00.615406 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5350fa2e-d877-447b-9984-c023f8f3152a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:00.616247 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350fa2e-d877-447b-9984-c023f8f3152a None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:00.616541 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350fa2e-d877-447b-9984-c023f8f3152a None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:00.616822 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350fa2e-d877-447b-9984-c023f8f3152a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:00.630821 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5350fa2e-d877-447b-9984-c023f8f3152a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:28:00.634641 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:00.640593 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-5350fa2e-d877-447b-9984-c023f8f3152a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:28:00.640593 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:00.640593 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:00.640593 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:00.640593 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:00.640593 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:00.640593 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:00.640593 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:00.640593 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:00.640593 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:00.640593 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:00.640593 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:00.640593 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:28:00.640593 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:28:00.640593 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:28:00.640593 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:28:00.641652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:28:00.641652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:28:00.641652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:28:00.641652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:28:00.641652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:00.641652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:00.641652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:00.641652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:00.641652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:00.641652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:00.641652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:00.641652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:28:00.641652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:00.641652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:00.641652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:00.641652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:00.642466 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:00.642466 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:00.642466 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:00.642466 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:00.642466 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:28:00.642466 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:28:00.642466 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:28:00.642466 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:28:00.642466 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:28:00.642466 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:28:00.642466 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:28:00.642466 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:28:00.646110 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 95/183] 10.222.0.36 () {56 vars in 1096 bytes} [Thu May 19 16:28:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:00.647524 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5350fa2e-d877-447b-9984-c023f8f3152a None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:00.677242 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:00.701188 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 89/184] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:28:00.701188 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf55ba62-e5b2-4849-9ab1-8ca58936af18 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:00.797415 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5350fa2e-d877-447b-9984-c023f8f3152a None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:00.813995 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf55ba62-e5b2-4849-9ab1-8ca58936af18 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:00.836125 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800af9d9-785e-4c9f-846b-911ecdde521b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:00.837113 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800af9d9-785e-4c9f-846b-911ecdde521b None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:00.837850 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800af9d9-785e-4c9f-846b-911ecdde521b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:00.838140 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800af9d9-785e-4c9f-846b-911ecdde521b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:00.839027 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800af9d9-785e-4c9f-846b-911ecdde521b None admin] RBAC: Adding query filter params (role.id=0e98fd94a6d24e46b9fe09a22a7df04a, scope.project.id=e2a683cd4fca4fe5ab8a9837a71d5988, user.id=a24c896dd82441dbb4ac8f0fab365a8f) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:00.839338 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800af9d9-785e-4c9f-846b-911ecdde521b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:00.840858 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800af9d9-785e-4c9f-846b-911ecdde521b None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:00.857187 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 96/185] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:28:00 2022] GET /identity/v3/role_assignments?user.id=a24c896dd82441dbb4ac8f0fab365a8f&scope.project.id=e2a683cd4fca4fe5ab8a9837a71d5988&role.id=0e98fd94a6d24e46b9fe09a22a7df04a => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:00.857187 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800af9d9-785e-4c9f-846b-911ecdde521b None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:00.867664 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d69735-05a6-495b-b88e-7a7d66e3d85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:00.868363 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d69735-05a6-495b-b88e-7a7d66e3d85d None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:00.868646 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d69735-05a6-495b-b88e-7a7d66e3d85d None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:00.868928 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d69735-05a6-495b-b88e-7a7d66e3d85d None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:00.869807 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d69735-05a6-495b-b88e-7a7d66e3d85d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:00.870123 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d69735-05a6-495b-b88e-7a7d66e3d85d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:00.871587 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d69735-05a6-495b-b88e-7a7d66e3d85d None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:00.882058 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 90/186] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:28:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:00.883366 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3bfc7-f717-4c69-ba04-68342daf9a73 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:00.883665 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3bfc7-f717-4c69-ba04-68342daf9a73 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:00.883940 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3bfc7-f717-4c69-ba04-68342daf9a73 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:00.895776 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 91/187] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:00 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:00.897467 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb3bfc7-f717-4c69-ba04-68342daf9a73 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:00.938028 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:00.978288 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:01.013803 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800af9d9-785e-4c9f-846b-911ecdde521b None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:01.041486 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb3bfc7-f717-4c69-ba04-68342daf9a73 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:01.085989 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9532a5-bfe4-42d0-896c-2ccf8b156a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:01.087199 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9532a5-bfe4-42d0-896c-2ccf8b156a64 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:01.087807 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9532a5-bfe4-42d0-896c-2ccf8b156a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:01.088241 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9532a5-bfe4-42d0-896c-2ccf8b156a64 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:01.099897 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48555c17-e74c-404a-9012-2518945a35e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:01.101450 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9532a5-bfe4-42d0-896c-2ccf8b156a64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:28:01.101603 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48555c17-e74c-404a-9012-2518945a35e4 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:01.101967 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48555c17-e74c-404a-9012-2518945a35e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:01.102465 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48555c17-e74c-404a-9012-2518945a35e4 None admin] PATH_INFO: `/v3/users/7654546351bf4ed18049c396e3d0a371` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:01.112210 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-7d9532a5-bfe4-42d0-896c-2ccf8b156a64 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 16:28:01.112210 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:01.112210 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:01.112210 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:01.112210 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:01.112210 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:01.112210 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:01.112210 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:01.112210 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:01.112210 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:01.112210 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:01.112210 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:01.112210 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:28:01.112210 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:28:01.112210 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:28:01.112210 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:28:01.113179 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:28:01.113179 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:28:01.113179 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:28:01.113179 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:28:01.113179 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:01.113179 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:01.113179 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:01.113179 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:01.113179 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:01.113179 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:01.113179 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:01.113179 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:28:01.113179 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:01.113179 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:01.113179 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:01.113179 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:01.113978 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:01.113978 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:01.113978 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:01.113978 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:01.113978 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:28:01.113978 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:28:01.113978 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:28:01.113978 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:28:01.113978 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:28:01.113978 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:28:01.113978 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 16:28:01.113978 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:28:01.116932 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 97/188] 10.222.0.36 () {56 vars in 1114 bytes} [Thu May 19 16:28:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:01.121087 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081b552-de6e-4c50-8045-5d22c15b7ffa None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:01.121644 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081b552-de6e-4c50-8045-5d22c15b7ffa None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:01.122154 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081b552-de6e-4c50-8045-5d22c15b7ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:01.125197 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48555c17-e74c-404a-9012-2518945a35e4 None admin] RBAC: Authorizing `identity:get_user(user_id=7654546351bf4ed18049c396e3d0a371)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:01.127012 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48555c17-e74c-404a-9012-2518945a35e4 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:01.129716 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 92/189] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:28:00 2022] GET /identity/v3/users/7654546351bf4ed18049c396e3d0a371 => generated 283 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:01.130516 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48555c17-e74c-404a-9012-2518945a35e4 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:01.162336 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-9081b552-de6e-4c50-8045-5d22c15b7ffa None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:01.163649 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-9081b552-de6e-4c50-8045-5d22c15b7ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:01.232057 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:01.241343 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 98/190] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:28:01.242651 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9081b552-de6e-4c50-8045-5d22c15b7ffa None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:01.254442 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48555c17-e74c-404a-9012-2518945a35e4 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:01.291132 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc425b-547d-4645-bc97-0412b4c99167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:01.291798 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc425b-547d-4645-bc97-0412b4c99167 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:01.292079 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc425b-547d-4645-bc97-0412b4c99167 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:01.292349 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc425b-547d-4645-bc97-0412b4c99167 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:01.293099 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcc425b-547d-4645-bc97-0412b4c99167 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:01.293680 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcc425b-547d-4645-bc97-0412b4c99167 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:01.294799 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcc425b-547d-4645-bc97-0412b4c99167 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:01.307679 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 93/191] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:28:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:01.307679 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc425b-547d-4645-bc97-0412b4c99167 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:01.336965 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:01.363539 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9081b552-de6e-4c50-8045-5d22c15b7ffa None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:01.400396 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea66375f-2bd1-4be4-8f3d-331de7c5a206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:01.401065 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea66375f-2bd1-4be4-8f3d-331de7c5a206 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:01.401368 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea66375f-2bd1-4be4-8f3d-331de7c5a206 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:01.401656 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea66375f-2bd1-4be4-8f3d-331de7c5a206 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:01.421511 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc425b-547d-4645-bc97-0412b4c99167 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:01.460621 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c58ef95-3595-453e-ae37-8b4c93696948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:01.461575 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58ef95-3595-453e-ae37-8b4c93696948 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:01.461866 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58ef95-3595-453e-ae37-8b4c93696948 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:01.462148 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58ef95-3595-453e-ae37-8b4c93696948 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:01.463770 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c58ef95-3595-453e-ae37-8b4c93696948 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:01.465054 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c58ef95-3595-453e-ae37-8b4c93696948 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:01.466726 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 94/192] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:28:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:01.466726 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c58ef95-3595-453e-ae37-8b4c93696948 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:01.480734 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:01.482017 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea66375f-2bd1-4be4-8f3d-331de7c5a206 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:01.483937 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea66375f-2bd1-4be4-8f3d-331de7c5a206 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:01.564982 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:01.576648 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c58ef95-3595-453e-ae37-8b4c93696948 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:01.599300 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 99/193] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:28:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 16:28:01.599300 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea66375f-2bd1-4be4-8f3d-331de7c5a206 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:01.616787 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997552b2-3e69-42fe-b4e0-3d7efbe3c88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:01.617716 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997552b2-3e69-42fe-b4e0-3d7efbe3c88b None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:01.618006 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997552b2-3e69-42fe-b4e0-3d7efbe3c88b None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:01.618288 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997552b2-3e69-42fe-b4e0-3d7efbe3c88b None admin] PATH_INFO: `/v3/users/7654546351bf4ed18049c396e3d0a371` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:01.619986 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997552b2-3e69-42fe-b4e0-3d7efbe3c88b None admin] RBAC: Authorizing `identity:get_user(user_id=7654546351bf4ed18049c396e3d0a371)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:01.621470 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997552b2-3e69-42fe-b4e0-3d7efbe3c88b None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:01.623199 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 95/194] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:28:01 2022] GET /identity/v3/users/7654546351bf4ed18049c396e3d0a371 => generated 283 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:01.628997 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997552b2-3e69-42fe-b4e0-3d7efbe3c88b None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:01.709460 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea66375f-2bd1-4be4-8f3d-331de7c5a206 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:01.716710 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:01.741580 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997552b2-3e69-42fe-b4e0-3d7efbe3c88b None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:01.756394 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925ffa0-8d7f-4915-888e-d7f091896dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:01.757080 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925ffa0-8d7f-4915-888e-d7f091896dfc None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:01.757399 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925ffa0-8d7f-4915-888e-d7f091896dfc None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:01.757733 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925ffa0-8d7f-4915-888e-d7f091896dfc None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:01.758448 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c925ffa0-8d7f-4915-888e-d7f091896dfc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:01.759918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c925ffa0-8d7f-4915-888e-d7f091896dfc None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:01.778907 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3be6e0-db3a-4839-948b-bb426e5b7a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:01.779845 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3be6e0-db3a-4839-948b-bb426e5b7a3f None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:01.780133 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3be6e0-db3a-4839-948b-bb426e5b7a3f None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:01.780438 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3be6e0-db3a-4839-948b-bb426e5b7a3f None admin] PATH_INFO: `/v3/projects/230decc93e3a483aa3d5cc33c0f662a9` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:01.781681 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3be6e0-db3a-4839-948b-bb426e5b7a3f None admin] RBAC: Authorizing `identity:get_project(project_id=230decc93e3a483aa3d5cc33c0f662a9)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:01.782229 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 100/195] 10.222.0.36 () {60 vars in 1132 bytes} [Thu May 19 16:28:01 2022] POST /identity/v3/users => generated 297 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:28:01.783137 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3be6e0-db3a-4839-948b-bb426e5b7a3f None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:01.790372 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 96/196] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:28:01 2022] GET /identity/v3/projects/230decc93e3a483aa3d5cc33c0f662a9 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:01.791375 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3be6e0-db3a-4839-948b-bb426e5b7a3f None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:01.791478 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925ffa0-8d7f-4915-888e-d7f091896dfc None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:01.838603 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:01.926106 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3be6e0-db3a-4839-948b-bb426e5b7a3f None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:01.944450 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925ffa0-8d7f-4915-888e-d7f091896dfc None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:01.977136 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461573e-8845-4582-9d5f-d1d98ca3ad90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:01.978938 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461573e-8845-4582-9d5f-d1d98ca3ad90 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:01.979723 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461573e-8845-4582-9d5f-d1d98ca3ad90 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:01.980368 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461573e-8845-4582-9d5f-d1d98ca3ad90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:01.981744 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e461573e-8845-4582-9d5f-d1d98ca3ad90 None admin] RBAC: Adding query filter params (role.id=e2915fe109244a07a9be96bd2b219fdd, scope.project.id=230decc93e3a483aa3d5cc33c0f662a9, user.id=7654546351bf4ed18049c396e3d0a371) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:01.982464 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e461573e-8845-4582-9d5f-d1d98ca3ad90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:01.983563 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1228b2b2-8e31-4686-b6cc-68e3b3a46daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:01.984230 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228b2b2-8e31-4686-b6cc-68e3b3a46daf None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:01.984518 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228b2b2-8e31-4686-b6cc-68e3b3a46daf None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:01.984796 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228b2b2-8e31-4686-b6cc-68e3b3a46daf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:01.985192 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e461573e-8845-4582-9d5f-d1d98ca3ad90 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:01.993873 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 97/197] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:28:01 2022] GET /identity/v3/role_assignments?user.id=7654546351bf4ed18049c396e3d0a371&scope.project.id=230decc93e3a483aa3d5cc33c0f662a9&role.id=e2915fe109244a07a9be96bd2b219fdd => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:02.094780 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1228b2b2-8e31-4686-b6cc-68e3b3a46daf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:02.096508 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1228b2b2-8e31-4686-b6cc-68e3b3a46daf None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:02.132769 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:02.219751 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 101/198] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:28:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 16:28:02.226162 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461573e-8845-4582-9d5f-d1d98ca3ad90 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:02.242818 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:02.251031 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c432c-e7e8-4185-9d89-001c8d9f4c9f None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:02.251663 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c432c-e7e8-4185-9d89-001c8d9f4c9f None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:02.252042 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c432c-e7e8-4185-9d89-001c8d9f4c9f None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:02.253758 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 102/199] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 16:28:02.259750 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e63bbd-8f3f-4538-a4d6-155276d1f8a0 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:02.260318 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e63bbd-8f3f-4538-a4d6-155276d1f8a0 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:02.260847 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e63bbd-8f3f-4538-a4d6-155276d1f8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:02.280232 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-26e63bbd-8f3f-4538-a4d6-155276d1f8a0 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:02.281692 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-26e63bbd-8f3f-4538-a4d6-155276d1f8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:02.351767 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461573e-8845-4582-9d5f-d1d98ca3ad90 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:02.359857 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 103/200] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:28:02.382872 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e63bbd-8f3f-4538-a4d6-155276d1f8a0 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:02.391205 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2399d631-7970-4fc4-a1cc-e2e8fc4c583f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:02.392433 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2399d631-7970-4fc4-a1cc-e2e8fc4c583f None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:02.392897 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2399d631-7970-4fc4-a1cc-e2e8fc4c583f None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:02.393366 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2399d631-7970-4fc4-a1cc-e2e8fc4c583f None admin] PATH_INFO: `/v3/projects/230decc93e3a483aa3d5cc33c0f662a9` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:02.395340 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2399d631-7970-4fc4-a1cc-e2e8fc4c583f None admin] RBAC: Authorizing `identity:get_project(project_id=230decc93e3a483aa3d5cc33c0f662a9)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:02.397366 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2399d631-7970-4fc4-a1cc-e2e8fc4c583f None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:02.399339 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 98/201] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:28:02 2022] GET /identity/v3/projects/230decc93e3a483aa3d5cc33c0f662a9 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:28:02.405260 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2399d631-7970-4fc4-a1cc-e2e8fc4c583f None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:02.453729 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:02.497980 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:02.526526 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2399d631-7970-4fc4-a1cc-e2e8fc4c583f None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:02.532167 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e63bbd-8f3f-4538-a4d6-155276d1f8a0 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:02.566386 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a7b2b8-9ae7-4f0d-bc20-5c2153668c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:02.567445 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7b2b8-9ae7-4f0d-bc20-5c2153668c72 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:02.567943 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7b2b8-9ae7-4f0d-bc20-5c2153668c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:02.568442 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7b2b8-9ae7-4f0d-bc20-5c2153668c72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:02.569551 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a7b2b8-9ae7-4f0d-bc20-5c2153668c72 None admin] RBAC: Adding query filter params (role.id=50f62a33eb434682baa7529619fe14b0, scope.project.id=230decc93e3a483aa3d5cc33c0f662a9, user.id=7654546351bf4ed18049c396e3d0a371) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:02.570196 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a7b2b8-9ae7-4f0d-bc20-5c2153668c72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:02.571963 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a7b2b8-9ae7-4f0d-bc20-5c2153668c72 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:02.578920 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 99/202] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:28:02 2022] GET /identity/v3/role_assignments?user.id=7654546351bf4ed18049c396e3d0a371&scope.project.id=230decc93e3a483aa3d5cc33c0f662a9&role.id=50f62a33eb434682baa7529619fe14b0 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:28:02.596888 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9a9ee0-546b-43dc-ac2b-c59035248926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:02.598115 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a9ee0-546b-43dc-ac2b-c59035248926 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:02.598753 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a9ee0-546b-43dc-ac2b-c59035248926 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:02.599252 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a9ee0-546b-43dc-ac2b-c59035248926 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:02.620430 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9a9ee0-546b-43dc-ac2b-c59035248926 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:28:02.634137 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-db9a9ee0-546b-43dc-ac2b-c59035248926 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 16:28:02.634137 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:02.634137 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:02.634137 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:02.634137 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:02.634137 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:02.634137 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:02.634137 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:02.634137 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:02.634137 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:02.634137 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:02.634137 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:02.634137 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:28:02.634137 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:28:02.634137 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:28:02.634137 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:28:02.635208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:28:02.635208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:28:02.635208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:28:02.635208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:28:02.635208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:02.635208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:02.635208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:02.635208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:02.635208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:02.635208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:02.635208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:02.635208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:28:02.635208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:02.635208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:02.635208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:02.635208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:02.636050 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:02.636050 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:02.636050 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:02.636050 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:02.636050 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:28:02.636050 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:28:02.636050 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:28:02.636050 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:28:02.636050 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:28:02.636050 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:28:02.636050 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 16:28:02.636050 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:28:02.638753 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 104/203] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:28:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:02.650518 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a7b2b8-9ae7-4f0d-bc20-5c2153668c72 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:02.783345 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a7b2b8-9ae7-4f0d-bc20-5c2153668c72 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:02.824880 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31fe56c-1092-4c06-a6ac-ec8d46b74488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:02.825966 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31fe56c-1092-4c06-a6ac-ec8d46b74488 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:02.826482 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31fe56c-1092-4c06-a6ac-ec8d46b74488 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:02.827013 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31fe56c-1092-4c06-a6ac-ec8d46b74488 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:02.828082 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31fe56c-1092-4c06-a6ac-ec8d46b74488 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:02.828602 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31fe56c-1092-4c06-a6ac-ec8d46b74488 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:02.830426 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31fe56c-1092-4c06-a6ac-ec8d46b74488 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:02.842802 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 100/204] 10.222.0.36 () {56 vars in 1101 bytes} [Thu May 19 16:28:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:02.848899 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9a9ee0-546b-43dc-ac2b-c59035248926 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:03.018529 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9a9ee0-546b-43dc-ac2b-c59035248926 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:03.046572 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:03.067496 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34824231-432e-4a66-8b65-3f000390738b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:03.068382 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34824231-432e-4a66-8b65-3f000390738b None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:03.068672 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34824231-432e-4a66-8b65-3f000390738b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:03.068955 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34824231-432e-4a66-8b65-3f000390738b None admin] PATH_INFO: `/v3/users/16bccb2c02f64fb0892b8adc61acbd33` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:03.073873 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34824231-432e-4a66-8b65-3f000390738b None admin] RBAC: Authorizing `identity:get_user(user_id=16bccb2c02f64fb0892b8adc61acbd33)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:03.075369 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34824231-432e-4a66-8b65-3f000390738b None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:03.077447 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 105/205] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:28:02 2022] GET /identity/v3/users/16bccb2c02f64fb0892b8adc61acbd33 => generated 289 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:03.082440 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31fe56c-1092-4c06-a6ac-ec8d46b74488 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:03.086570 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:03.219633 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31fe56c-1092-4c06-a6ac-ec8d46b74488 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:03.256079 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9bd5b-4417-4e9f-867f-d3a7894b54d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:03.257122 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9bd5b-4417-4e9f-867f-d3a7894b54d1 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:03.257630 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9bd5b-4417-4e9f-867f-d3a7894b54d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:03.258097 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9bd5b-4417-4e9f-867f-d3a7894b54d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:03.332947 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df9bd5b-4417-4e9f-867f-d3a7894b54d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:03.334754 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df9bd5b-4417-4e9f-867f-d3a7894b54d1 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:03.337953 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:03.452160 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 101/206] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:28:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 16:28:03.459475 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34824231-432e-4a66-8b65-3f000390738b None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:03.606282 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34824231-432e-4a66-8b65-3f000390738b None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:03.654474 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d00b6-5f37-4b3e-a939-bd762c7d4056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:03.655363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d00b6-5f37-4b3e-a939-bd762c7d4056 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:03.655655 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d00b6-5f37-4b3e-a939-bd762c7d4056 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:03.655939 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d00b6-5f37-4b3e-a939-bd762c7d4056 None admin] PATH_INFO: `/v3/projects/d4a456cd7f83414a895ab7982fec2a68` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:03.661923 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9d00b6-5f37-4b3e-a939-bd762c7d4056 None admin] RBAC: Authorizing `identity:get_project(project_id=d4a456cd7f83414a895ab7982fec2a68)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:03.663483 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9d00b6-5f37-4b3e-a939-bd762c7d4056 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:03.665282 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 106/207] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:28:03 2022] GET /identity/v3/projects/d4a456cd7f83414a895ab7982fec2a68 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:03.671015 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9bd5b-4417-4e9f-867f-d3a7894b54d1 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:03.800594 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9bd5b-4417-4e9f-867f-d3a7894b54d1 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:03.803426 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:03.843175 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d3d5dd-903d-4d09-9ae6-6e93af35d56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:03.844255 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d3d5dd-903d-4d09-9ae6-6e93af35d56b None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:03.844847 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d3d5dd-903d-4d09-9ae6-6e93af35d56b None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:03.845348 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d3d5dd-903d-4d09-9ae6-6e93af35d56b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:03.846389 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d3d5dd-903d-4d09-9ae6-6e93af35d56b None admin] RBAC: Adding query filter params (role.id=156d234cfe9d4844933886f1ba568694, scope.project.id=d4a456cd7f83414a895ab7982fec2a68, user.id=16bccb2c02f64fb0892b8adc61acbd33) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:03.847536 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d3d5dd-903d-4d09-9ae6-6e93af35d56b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:03.849352 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d3d5dd-903d-4d09-9ae6-6e93af35d56b None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:03.856397 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 102/208] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:28:03 2022] GET /identity/v3/role_assignments?user.id=16bccb2c02f64fb0892b8adc61acbd33&scope.project.id=d4a456cd7f83414a895ab7982fec2a68&role.id=156d234cfe9d4844933886f1ba568694 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:04.661696 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8037d8de-0ab6-4e5b-bbdd-dbd76b579a94 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:04.662220 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8037d8de-0ab6-4e5b-bbdd-dbd76b579a94 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:04.662899 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8037d8de-0ab6-4e5b-bbdd-dbd76b579a94 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:04.665862 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 107/209] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:04.671647 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c45f69-395a-4ae5-8dee-026772d49c73 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:04.672513 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c45f69-395a-4ae5-8dee-026772d49c73 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:04.672992 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c45f69-395a-4ae5-8dee-026772d49c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:04.692170 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-c1c45f69-395a-4ae5-8dee-026772d49c73 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:04.693770 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-c1c45f69-395a-4ae5-8dee-026772d49c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:04.696909 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3abc87-ca19-4b64-90ae-983d1aeaeb3f None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:04.697216 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3abc87-ca19-4b64-90ae-983d1aeaeb3f None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:04.697501 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3abc87-ca19-4b64-90ae-983d1aeaeb3f None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:04.702085 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 108/210] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 16:28:04.704468 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d5768-eec9-4e2c-8ad4-68a4c1f69570 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:04.704761 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d5768-eec9-4e2c-8ad4-68a4c1f69570 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:04.705043 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d5768-eec9-4e2c-8ad4-68a4c1f69570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:04.724004 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-a52d5768-eec9-4e2c-8ad4-68a4c1f69570 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:04.725582 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-a52d5768-eec9-4e2c-8ad4-68a4c1f69570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:04.784514 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:04.797342 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 103/211] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:28:04.814431 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 109/212] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 16:28:04.819649 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c45f69-395a-4ae5-8dee-026772d49c73 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:04.842275 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52d5768-eec9-4e2c-8ad4-68a4c1f69570 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:04.937330 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:04.942802 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c45f69-395a-4ae5-8dee-026772d49c73 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:04.973863 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52d5768-eec9-4e2c-8ad4-68a4c1f69570 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:04.992434 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1212e435-932c-4737-b40b-1a3849d834f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:04.993706 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1212e435-932c-4737-b40b-1a3849d834f4 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:04.994205 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1212e435-932c-4737-b40b-1a3849d834f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:04.994917 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1212e435-932c-4737-b40b-1a3849d834f4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:05.008508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1212e435-932c-4737-b40b-1a3849d834f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:28:05.017357 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-1212e435-932c-4737-b40b-1a3849d834f4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 16:28:05.017357 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:05.017357 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:05.017357 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:05.017357 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:05.017357 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:05.017357 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:05.017357 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:05.017357 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:05.017357 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:05.017357 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:05.017357 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:05.017357 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:28:05.017357 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:28:05.017357 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:28:05.017357 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:28:05.018344 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:28:05.018344 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:28:05.018344 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:28:05.018344 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:28:05.018344 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:05.018344 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:05.018344 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:05.018344 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:05.018344 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:05.018344 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:05.018344 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:05.018344 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:05.018344 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:05.018344 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:05.018344 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:05.018344 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:05.019140 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:05.019140 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:05.019140 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:05.019140 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:05.019140 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:28:05.019140 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:28:05.019140 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:28:05.019140 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:28:05.019140 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:28:05.019140 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:28:05.019140 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 16:28:05.019140 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:05.022985 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 104/213] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:28:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:05.028990 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1212e435-932c-4737-b40b-1a3849d834f4 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:05.029851 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45c3b48-eb03-4239-a4f3-03d356df8798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:05.030908 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c3b48-eb03-4239-a4f3-03d356df8798 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:05.031485 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c3b48-eb03-4239-a4f3-03d356df8798 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:05.031958 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c3b48-eb03-4239-a4f3-03d356df8798 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:05.054554 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45c3b48-eb03-4239-a4f3-03d356df8798 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:28:05.062749 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-c45c3b48-eb03-4239-a4f3-03d356df8798 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:28:05.062749 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:05.062749 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:05.062749 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:05.062749 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:05.062749 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:05.062749 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:05.062749 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:05.062749 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:05.062749 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:05.062749 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:05.062749 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:05.062749 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:28:05.062749 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:28:05.062749 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:28:05.062749 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:28:05.063667 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:28:05.063667 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:28:05.063667 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:28:05.063667 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:28:05.063667 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:05.063667 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:05.063667 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:05.063667 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:05.063667 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:05.063667 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:05.063667 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:05.063667 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:28:05.063667 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:05.063667 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:05.063667 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:05.063667 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:05.064457 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:05.064457 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:05.064457 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:05.064457 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:05.064457 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:28:05.064457 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:28:05.064457 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:28:05.064457 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:28:05.064457 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:28:05.064457 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:28:05.064457 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:28:05.064457 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:28:05.066998 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 110/214] 10.222.0.36 () {56 vars in 1096 bytes} [Thu May 19 16:28:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 16:28:05.077789 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45c3b48-eb03-4239-a4f3-03d356df8798 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:05.151106 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:05.178665 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1212e435-932c-4737-b40b-1a3849d834f4 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:05.223388 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45c3b48-eb03-4239-a4f3-03d356df8798 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:05.226715 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8726d0-2270-4a5b-b772-665527d2f809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:05.228107 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8726d0-2270-4a5b-b772-665527d2f809 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:05.228107 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8726d0-2270-4a5b-b772-665527d2f809 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:05.228107 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8726d0-2270-4a5b-b772-665527d2f809 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:05.229294 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8726d0-2270-4a5b-b772-665527d2f809 None admin] RBAC: Adding query filter params (name=member) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:05.229294 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8726d0-2270-4a5b-b772-665527d2f809 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:05.230543 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8726d0-2270-4a5b-b772-665527d2f809 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:05.245563 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 105/215] 10.222.0.36 () {56 vars in 1101 bytes} [Thu May 19 16:28:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:28:05.245563 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d441126-e90b-43ec-90d4-43bd601c28e9 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:05.245563 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d441126-e90b-43ec-90d4-43bd601c28e9 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:05.245563 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d441126-e90b-43ec-90d4-43bd601c28e9 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:05.245563 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 106/216] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:05.249179 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d441126-e90b-43ec-90d4-43bd601c28e9 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:05.274430 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27ad8b4-be6b-4dec-a010-2ddd42d3f432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:05.275576 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27ad8b4-be6b-4dec-a010-2ddd42d3f432 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:05.275886 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27ad8b4-be6b-4dec-a010-2ddd42d3f432 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:05.276170 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27ad8b4-be6b-4dec-a010-2ddd42d3f432 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:05.277110 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:05.277575 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27ad8b4-be6b-4dec-a010-2ddd42d3f432 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:05.277907 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27ad8b4-be6b-4dec-a010-2ddd42d3f432 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:05.279619 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27ad8b4-be6b-4dec-a010-2ddd42d3f432 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:05.293193 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 111/217] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:28:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:28:05.293193 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b14ab3-759b-43f7-87c6-9373d8ef75b2 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:05.293193 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b14ab3-759b-43f7-87c6-9373d8ef75b2 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:05.293193 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b14ab3-759b-43f7-87c6-9373d8ef75b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:05.311708 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-77b14ab3-759b-43f7-87c6-9373d8ef75b2 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:05.313818 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-77b14ab3-759b-43f7-87c6-9373d8ef75b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:05.375411 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d441126-e90b-43ec-90d4-43bd601c28e9 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:05.398653 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 112/218] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:28:05.399656 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b14ab3-759b-43f7-87c6-9373d8ef75b2 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:05.417215 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e596ef16-fc61-4cc7-a2fb-4503883129a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:05.418107 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596ef16-fc61-4cc7-a2fb-4503883129a2 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:05.418388 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596ef16-fc61-4cc7-a2fb-4503883129a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:05.418665 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596ef16-fc61-4cc7-a2fb-4503883129a2 None admin] PATH_INFO: `/v3/users/c068746943354012a817dc656ba0a7af` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:05.434647 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e596ef16-fc61-4cc7-a2fb-4503883129a2 None admin] RBAC: Authorizing `identity:get_user(user_id=c068746943354012a817dc656ba0a7af)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:05.436123 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e596ef16-fc61-4cc7-a2fb-4503883129a2 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:05.437939 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 107/219] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:28:05 2022] GET /identity/v3/users/c068746943354012a817dc656ba0a7af => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:05.442703 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09a0fc-6808-42fc-80da-e427f6c4d57f None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:05.442703 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09a0fc-6808-42fc-80da-e427f6c4d57f None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:05.442703 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09a0fc-6808-42fc-80da-e427f6c4d57f None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:05.442703 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 108/220] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:05.442703 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e09a0fc-6808-42fc-80da-e427f6c4d57f None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:05.447164 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:05.520661 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b14ab3-759b-43f7-87c6-9373d8ef75b2 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:05.537134 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:05.561365 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e09a0fc-6808-42fc-80da-e427f6c4d57f None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:05.564317 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aff888-eaaa-4b78-83f8-15cb7abadf01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:05.565257 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aff888-eaaa-4b78-83f8-15cb7abadf01 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:05.565544 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aff888-eaaa-4b78-83f8-15cb7abadf01 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:05.565823 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aff888-eaaa-4b78-83f8-15cb7abadf01 None admin] PATH_INFO: `/v3/users/7654546351bf4ed18049c396e3d0a371` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:05.567040 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aff888-eaaa-4b78-83f8-15cb7abadf01 None admin] RBAC: Authorizing `identity:get_user(user_id=7654546351bf4ed18049c396e3d0a371)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:05.568565 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aff888-eaaa-4b78-83f8-15cb7abadf01 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:05.570253 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 113/221] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:28:05 2022] GET /identity/v3/users/7654546351bf4ed18049c396e3d0a371 => generated 283 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:05.570253 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aff888-eaaa-4b78-83f8-15cb7abadf01 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:05.604734 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0330f143-6a56-469d-af47-8b129f2d76aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:05.605631 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0330f143-6a56-469d-af47-8b129f2d76aa None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:05.605844 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0330f143-6a56-469d-af47-8b129f2d76aa None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:05.606120 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0330f143-6a56-469d-af47-8b129f2d76aa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:05.607282 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0330f143-6a56-469d-af47-8b129f2d76aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:05.608738 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0330f143-6a56-469d-af47-8b129f2d76aa None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:05.610598 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 109/222] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:28:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:05.617456 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5382c0-5277-4588-9d16-d1ab29f2f7e7 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:05.617744 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5382c0-5277-4588-9d16-d1ab29f2f7e7 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:05.618025 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5382c0-5277-4588-9d16-d1ab29f2f7e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:05.634740 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-2c5382c0-5277-4588-9d16-d1ab29f2f7e7 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:05.635982 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-2c5382c0-5277-4588-9d16-d1ab29f2f7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:05.684584 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aff888-eaaa-4b78-83f8-15cb7abadf01 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:05.691910 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:05.705599 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 110/223] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:28:05.717779 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5382c0-5277-4588-9d16-d1ab29f2f7e7 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:05.720672 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc3eb5e-1d9b-436e-bb03-ae0ab06d5096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:05.722010 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3eb5e-1d9b-436e-bb03-ae0ab06d5096 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:05.722322 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3eb5e-1d9b-436e-bb03-ae0ab06d5096 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:05.722599 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3eb5e-1d9b-436e-bb03-ae0ab06d5096 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:05.784391 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:05.790393 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc3eb5e-1d9b-436e-bb03-ae0ab06d5096 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:05.791874 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc3eb5e-1d9b-436e-bb03-ae0ab06d5096 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:05.812903 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5382c0-5277-4588-9d16-d1ab29f2f7e7 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:05.858479 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38e9f59-4c86-43fa-a9fc-5fb90f1c8e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:05.859439 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e9f59-4c86-43fa-a9fc-5fb90f1c8e23 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:05.859745 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e9f59-4c86-43fa-a9fc-5fb90f1c8e23 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:05.860026 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e9f59-4c86-43fa-a9fc-5fb90f1c8e23 None admin] PATH_INFO: `/v3/projects/230decc93e3a483aa3d5cc33c0f662a9` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:05.861417 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38e9f59-4c86-43fa-a9fc-5fb90f1c8e23 None admin] RBAC: Authorizing `identity:get_project(project_id=230decc93e3a483aa3d5cc33c0f662a9)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:05.863567 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38e9f59-4c86-43fa-a9fc-5fb90f1c8e23 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:05.866137 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 111/224] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:28:05 2022] GET /identity/v3/projects/230decc93e3a483aa3d5cc33c0f662a9 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:05.866321 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38e9f59-4c86-43fa-a9fc-5fb90f1c8e23 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:05.918501 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 114/225] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:28:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 16:28:05.919199 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc3eb5e-1d9b-436e-bb03-ae0ab06d5096 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:05.929114 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:05.990016 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38e9f59-4c86-43fa-a9fc-5fb90f1c8e23 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:06.024624 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1932561f-56d4-4dc3-8d13-8440d5306306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:06.025409 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932561f-56d4-4dc3-8d13-8440d5306306 None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:06.025712 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932561f-56d4-4dc3-8d13-8440d5306306 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:06.026001 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932561f-56d4-4dc3-8d13-8440d5306306 None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:06.026742 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1932561f-56d4-4dc3-8d13-8440d5306306 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:06.028203 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1932561f-56d4-4dc3-8d13-8440d5306306 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:06.046149 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:06.051039 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 112/226] 10.222.0.36 () {60 vars in 1132 bytes} [Thu May 19 16:28:05 2022] POST /identity/v3/users => generated 297 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:28:06.052178 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1932561f-56d4-4dc3-8d13-8440d5306306 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:06.070850 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc3eb5e-1d9b-436e-bb03-ae0ab06d5096 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:06.139672 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4fe0f8-de43-4fd4-a6d7-7461b2d0a896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:06.140640 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4fe0f8-de43-4fd4-a6d7-7461b2d0a896 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:06.141144 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4fe0f8-de43-4fd4-a6d7-7461b2d0a896 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:06.141664 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4fe0f8-de43-4fd4-a6d7-7461b2d0a896 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:06.162589 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4fe0f8-de43-4fd4-a6d7-7461b2d0a896 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:28:06.176819 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-cd4fe0f8-de43-4fd4-a6d7-7461b2d0a896 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 16:28:06.176819 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:06.176819 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:06.176819 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:06.176819 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:06.176819 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:06.176819 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:06.176819 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:06.176819 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:06.176819 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:06.176819 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:06.176819 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:06.176819 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:28:06.176819 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:28:06.176819 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:28:06.176819 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:28:06.177992 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:28:06.177992 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:28:06.177992 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:28:06.177992 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:28:06.177992 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:06.177992 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:06.177992 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:06.177992 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:06.177992 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:06.177992 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:06.177992 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:06.177992 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:28:06.177992 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:06.177992 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:06.177992 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:06.177992 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:06.178992 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:06.178992 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:06.178992 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:06.178992 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:06.178992 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:28:06.178992 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:28:06.178992 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:28:06.178992 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:28:06.178992 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:28:06.178992 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:28:06.178992 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 16:28:06.178992 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:28:06.182431 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 115/227] 10.222.0.36 () {56 vars in 1114 bytes} [Thu May 19 16:28:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:06.183649 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4fe0f8-de43-4fd4-a6d7-7461b2d0a896 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:06.214773 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1932561f-56d4-4dc3-8d13-8440d5306306 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:06.261293 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfbf1c0-924a-433f-a7db-46f5b835c58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:06.262226 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfbf1c0-924a-433f-a7db-46f5b835c58b None admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:06.262518 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfbf1c0-924a-433f-a7db-46f5b835c58b None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:06.262819 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfbf1c0-924a-433f-a7db-46f5b835c58b None admin] PATH_INFO: `/v3/projects/230decc93e3a483aa3d5cc33c0f662a9/users/7654546351bf4ed18049c396e3d0a371/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:06.264940 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfbf1c0-924a-433f-a7db-46f5b835c58b None admin] RBAC: Authorizing `identity:create_grant(project_id=230decc93e3a483aa3d5cc33c0f662a9, user_id=7654546351bf4ed18049c396e3d0a371, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:06.265680 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:06.266488 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfbf1c0-924a-433f-a7db-46f5b835c58b None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:06.285735 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 113/228] 10.222.0.36 () {58 vars in 1439 bytes} [Thu May 19 16:28:06 2022] PUT /identity/v3/projects/230decc93e3a483aa3d5cc33c0f662a9/users/7654546351bf4ed18049c396e3d0a371/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:28:06.294120 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfbf1c0-924a-433f-a7db-46f5b835c58b None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:06.327295 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4fe0f8-de43-4fd4-a6d7-7461b2d0a896 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:06.385943 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:06.388221 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2de16c4-0e2e-4e96-b2e9-8ee6f8473718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:06.389276 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de16c4-0e2e-4e96-b2e9-8ee6f8473718 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:06.389603 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de16c4-0e2e-4e96-b2e9-8ee6f8473718 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:06.390484 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de16c4-0e2e-4e96-b2e9-8ee6f8473718 None admin] PATH_INFO: `/v3/projects/230decc93e3a483aa3d5cc33c0f662a9` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:06.391323 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2de16c4-0e2e-4e96-b2e9-8ee6f8473718 None admin] RBAC: Authorizing `identity:get_project(project_id=230decc93e3a483aa3d5cc33c0f662a9)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:06.392995 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2de16c4-0e2e-4e96-b2e9-8ee6f8473718 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:06.395653 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 116/229] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:28:06 2022] GET /identity/v3/projects/230decc93e3a483aa3d5cc33c0f662a9 => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:06.402948 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2de16c4-0e2e-4e96-b2e9-8ee6f8473718 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:06.447721 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfbf1c0-924a-433f-a7db-46f5b835c58b None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:06.498179 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c82cac7-7d95-4858-b54f-40e76a5a3a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:06.498879 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c82cac7-7d95-4858-b54f-40e76a5a3a62 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:06.499173 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c82cac7-7d95-4858-b54f-40e76a5a3a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:06.499460 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c82cac7-7d95-4858-b54f-40e76a5a3a62 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:06.500247 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c82cac7-7d95-4858-b54f-40e76a5a3a62 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:06.500572 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c82cac7-7d95-4858-b54f-40e76a5a3a62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:06.502078 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c82cac7-7d95-4858-b54f-40e76a5a3a62 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:06.515462 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 114/230] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:28:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:06.522839 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c82cac7-7d95-4858-b54f-40e76a5a3a62 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:06.538841 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2de16c4-0e2e-4e96-b2e9-8ee6f8473718 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:06.590768 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a575b817-f5f0-486d-866e-3c2af34c47b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:06.592138 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a575b817-f5f0-486d-866e-3c2af34c47b6 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:06.592656 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a575b817-f5f0-486d-866e-3c2af34c47b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:06.593582 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a575b817-f5f0-486d-866e-3c2af34c47b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:06.595187 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a575b817-f5f0-486d-866e-3c2af34c47b6 None admin] RBAC: Adding query filter params (role.id=0e98fd94a6d24e46b9fe09a22a7df04a, scope.project.id=230decc93e3a483aa3d5cc33c0f662a9, user.id=c068746943354012a817dc656ba0a7af) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:06.596085 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a575b817-f5f0-486d-866e-3c2af34c47b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:06.598648 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a575b817-f5f0-486d-866e-3c2af34c47b6 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:06.601626 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:06.610351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 117/231] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:28:06 2022] GET /identity/v3/role_assignments?user.id=c068746943354012a817dc656ba0a7af&scope.project.id=230decc93e3a483aa3d5cc33c0f662a9&role.id=0e98fd94a6d24e46b9fe09a22a7df04a => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:06.662802 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c82cac7-7d95-4858-b54f-40e76a5a3a62 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:06.689731 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:06.704531 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c55093-687b-4086-860e-7c87fe1861c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:06.705899 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c55093-687b-4086-860e-7c87fe1861c2 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:06.706836 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c55093-687b-4086-860e-7c87fe1861c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:06.707321 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c55093-687b-4086-860e-7c87fe1861c2 None admin] PATH_INFO: `/v3/users/7654546351bf4ed18049c396e3d0a371` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:06.708923 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c55093-687b-4086-860e-7c87fe1861c2 None admin] RBAC: Authorizing `identity:get_user(user_id=7654546351bf4ed18049c396e3d0a371)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:06.710677 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c55093-687b-4086-860e-7c87fe1861c2 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:06.712614 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 115/232] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:28:06 2022] GET /identity/v3/users/7654546351bf4ed18049c396e3d0a371 => generated 283 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:06.718503 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a575b817-f5f0-486d-866e-3c2af34c47b6 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:06.851198 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a575b817-f5f0-486d-866e-3c2af34c47b6 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:06.899556 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b64cfde-f9b5-4f64-9e30-b29b2535b3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:06.900506 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64cfde-f9b5-4f64-9e30-b29b2535b3a8 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:06.900800 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64cfde-f9b5-4f64-9e30-b29b2535b3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:06.901085 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64cfde-f9b5-4f64-9e30-b29b2535b3a8 None admin] PATH_INFO: `/v3/projects/230decc93e3a483aa3d5cc33c0f662a9` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:06.902399 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b64cfde-f9b5-4f64-9e30-b29b2535b3a8 None admin] RBAC: Authorizing `identity:get_project(project_id=230decc93e3a483aa3d5cc33c0f662a9)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:06.903905 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b64cfde-f9b5-4f64-9e30-b29b2535b3a8 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:06.906059 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 118/233] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:28:06 2022] GET /identity/v3/projects/230decc93e3a483aa3d5cc33c0f662a9 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:06.911746 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c55093-687b-4086-860e-7c87fe1861c2 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:07.034475 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c55093-687b-4086-860e-7c87fe1861c2 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:07.085870 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e3fc0-8f90-4e6d-a740-96b9544117b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:07.086834 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e3fc0-8f90-4e6d-a740-96b9544117b4 None admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:07.087128 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e3fc0-8f90-4e6d-a740-96b9544117b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:07.087412 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e3fc0-8f90-4e6d-a740-96b9544117b4 None admin] PATH_INFO: `/v3/projects/230decc93e3a483aa3d5cc33c0f662a9/users/7654546351bf4ed18049c396e3d0a371/roles/50f62a33eb434682baa7529619fe14b0` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:07.093602 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12e3fc0-8f90-4e6d-a740-96b9544117b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=230decc93e3a483aa3d5cc33c0f662a9, user_id=7654546351bf4ed18049c396e3d0a371, role_id=50f62a33eb434682baa7529619fe14b0)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:07.095180 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12e3fc0-8f90-4e6d-a740-96b9544117b4 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:07.120446 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 116/234] 10.222.0.36 () {58 vars in 1439 bytes} [Thu May 19 16:28:06 2022] PUT /identity/v3/projects/230decc93e3a483aa3d5cc33c0f662a9/users/7654546351bf4ed18049c396e3d0a371/roles/50f62a33eb434682baa7529619fe14b0 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:28:08.816274 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712711f-4ded-417d-8b03-e9119cb856a8 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:08.816785 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712711f-4ded-417d-8b03-e9119cb856a8 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:08.817010 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712711f-4ded-417d-8b03-e9119cb856a8 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:08.818483 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 119/235] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:08.823999 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addafc28-1257-4c44-9d2e-30bb23f1a1f6 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:08.824437 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addafc28-1257-4c44-9d2e-30bb23f1a1f6 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:08.824630 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addafc28-1257-4c44-9d2e-30bb23f1a1f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:08.851351 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-addafc28-1257-4c44-9d2e-30bb23f1a1f6 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:08.852564 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-addafc28-1257-4c44-9d2e-30bb23f1a1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:08.934125 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:08.970156 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 117/236] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:28:09.006073 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d712711f-4ded-417d-8b03-e9119cb856a8 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:09.100547 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:09.153353 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d712711f-4ded-417d-8b03-e9119cb856a8 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:09.165008 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03125a-bdd3-42c1-91cf-6e1d037d109d None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:09.165535 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03125a-bdd3-42c1-91cf-6e1d037d109d None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:09.165899 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03125a-bdd3-42c1-91cf-6e1d037d109d None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:09.167462 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 118/237] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:09.174770 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9bf5ec-8131-4f62-8aee-c87df4114f36 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:09.175377 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9bf5ec-8131-4f62-8aee-c87df4114f36 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:09.175712 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9bf5ec-8131-4f62-8aee-c87df4114f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:09.212478 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-5e9bf5ec-8131-4f62-8aee-c87df4114f36 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:09.213743 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-5e9bf5ec-8131-4f62-8aee-c87df4114f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:09.220076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b84bf8-c485-4a51-9a32-e73025cd57b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:09.220889 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b84bf8-c485-4a51-9a32-e73025cd57b0 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:09.221194 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b84bf8-c485-4a51-9a32-e73025cd57b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:09.222052 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b84bf8-c485-4a51-9a32-e73025cd57b0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:09.241916 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b84bf8-c485-4a51-9a32-e73025cd57b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:28:09.250336 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-c2b84bf8-c485-4a51-9a32-e73025cd57b0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 16:28:09.250336 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:09.250336 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:09.250336 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:09.250336 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:09.250336 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:09.250336 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:09.250336 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:09.250336 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:09.250336 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:09.250336 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:09.250336 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:09.250336 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:28:09.250336 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:28:09.250336 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:28:09.250336 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:28:09.251325 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:28:09.251325 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:28:09.251325 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:28:09.251325 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:28:09.251325 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:09.251325 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:09.251325 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:09.251325 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:09.251325 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:09.251325 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:09.251325 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:09.251325 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:28:09.251325 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:09.251325 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:09.251325 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:09.251325 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:09.252141 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:09.252141 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:09.252141 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:09.252141 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:09.252141 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:28:09.252141 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:28:09.252141 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:28:09.252141 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:28:09.252141 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:28:09.252141 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:28:09.252141 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 16:28:09.252141 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:28:09.255682 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 120/238] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:28:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:09.261738 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b84bf8-c485-4a51-9a32-e73025cd57b0 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:09.363258 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 119/239] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:28:09.415607 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9bf5ec-8131-4f62-8aee-c87df4114f36 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:09.460258 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b84bf8-c485-4a51-9a32-e73025cd57b0 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:09.504337 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:09.515684 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf477eb-bf8e-4a9a-9af5-3eee5a30ad5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:09.516711 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf477eb-bf8e-4a9a-9af5-3eee5a30ad5a None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:09.517225 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf477eb-bf8e-4a9a-9af5-3eee5a30ad5a None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:09.517703 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf477eb-bf8e-4a9a-9af5-3eee5a30ad5a None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:09.519312 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf477eb-bf8e-4a9a-9af5-3eee5a30ad5a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:09.519832 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf477eb-bf8e-4a9a-9af5-3eee5a30ad5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:09.521729 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf477eb-bf8e-4a9a-9af5-3eee5a30ad5a None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:09.531476 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 121/240] 10.222.0.36 () {56 vars in 1101 bytes} [Thu May 19 16:28:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:28:09.537276 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf9ae32-7cf7-4d3b-be78-bcfec22b6ad7 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:09.537958 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf9ae32-7cf7-4d3b-be78-bcfec22b6ad7 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:09.538429 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf9ae32-7cf7-4d3b-be78-bcfec22b6ad7 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:09.540159 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 122/241] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:09.541980 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf9ae32-7cf7-4d3b-be78-bcfec22b6ad7 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:09.579632 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9bf5ec-8131-4f62-8aee-c87df4114f36 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:09.592370 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:09.625644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9598415d-0904-4abf-bec1-28352795f331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:09.627245 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598415d-0904-4abf-bec1-28352795f331 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:09.627857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598415d-0904-4abf-bec1-28352795f331 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:09.628463 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598415d-0904-4abf-bec1-28352795f331 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:09.642808 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9598415d-0904-4abf-bec1-28352795f331 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:28:09.652366 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-9598415d-0904-4abf-bec1-28352795f331 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:28:09.652366 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:09.652366 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:09.652366 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:09.652366 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:09.652366 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:09.652366 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:09.652366 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:09.652366 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:09.652366 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:09.652366 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:09.652366 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:09.652366 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:28:09.652366 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:28:09.652366 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:28:09.652366 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:28:09.653357 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:28:09.653357 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:28:09.653357 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:28:09.653357 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:28:09.653357 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:09.653357 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:09.653357 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:09.653357 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:09.653357 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:09.653357 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:09.653357 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:09.653357 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:09.653357 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:09.653357 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:09.653357 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:09.653357 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:09.654147 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:09.654147 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:09.654147 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:09.654147 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:09.654147 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:28:09.654147 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:28:09.654147 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:28:09.654147 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:28:09.654147 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:28:09.654147 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:28:09.654147 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:28:09.654147 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:09.656846 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 120/242] 10.222.0.36 () {56 vars in 1096 bytes} [Thu May 19 16:28:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 16:28:09.658585 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19687f00-5e34-446a-adc5-b9503f9ebc81 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:09.659628 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19687f00-5e34-446a-adc5-b9503f9ebc81 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:09.660082 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19687f00-5e34-446a-adc5-b9503f9ebc81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:09.671402 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf9ae32-7cf7-4d3b-be78-bcfec22b6ad7 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:09.679130 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-19687f00-5e34-446a-adc5-b9503f9ebc81 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:09.680629 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-19687f00-5e34-446a-adc5-b9503f9ebc81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:09.708383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d0b43-c765-4a4f-8a80-ebf31034e54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:09.709431 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d0b43-c765-4a4f-8a80-ebf31034e54a None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:09.709793 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d0b43-c765-4a4f-8a80-ebf31034e54a None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:09.710145 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d0b43-c765-4a4f-8a80-ebf31034e54a None admin] PATH_INFO: `/v3/users/24b89cd5cc53406db6fdbcbe2b60ad7b` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:09.727332 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3d0b43-c765-4a4f-8a80-ebf31034e54a None admin] RBAC: Authorizing `identity:get_user(user_id=24b89cd5cc53406db6fdbcbe2b60ad7b)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:09.729058 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3d0b43-c765-4a4f-8a80-ebf31034e54a None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:09.731053 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 123/243] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:28:09 2022] GET /identity/v3/users/24b89cd5cc53406db6fdbcbe2b60ad7b => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:28:09.736468 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d0b43-c765-4a4f-8a80-ebf31034e54a None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:09.759989 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 121/244] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:28:09.766570 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19687f00-5e34-446a-adc5-b9503f9ebc81 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:09.794758 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:09.856355 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d0b43-c765-4a4f-8a80-ebf31034e54a None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:09.895293 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb714bb-2cbf-4b53-a8b6-69b228161b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:09.896038 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb714bb-2cbf-4b53-a8b6-69b228161b6c None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:09.896609 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb714bb-2cbf-4b53-a8b6-69b228161b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:09.896882 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb714bb-2cbf-4b53-a8b6-69b228161b6c None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:09.897701 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb714bb-2cbf-4b53-a8b6-69b228161b6c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:09.898025 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb714bb-2cbf-4b53-a8b6-69b228161b6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:09.899444 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb714bb-2cbf-4b53-a8b6-69b228161b6c None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:09.904435 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:09.908647 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19687f00-5e34-446a-adc5-b9503f9ebc81 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:09.908793 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 124/245] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:28:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:09.914599 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb714bb-2cbf-4b53-a8b6-69b228161b6c None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:09.949969 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca357e-c160-488a-80d3-13ec82ba1559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:09.950632 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca357e-c160-488a-80d3-13ec82ba1559 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:09.950921 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca357e-c160-488a-80d3-13ec82ba1559 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:09.951201 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca357e-c160-488a-80d3-13ec82ba1559 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:10.021428 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb714bb-2cbf-4b53-a8b6-69b228161b6c None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:10.021880 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ca357e-c160-488a-80d3-13ec82ba1559 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:10.023374 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ca357e-c160-488a-80d3-13ec82ba1559 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:10.030793 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:10.056913 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d193c2c1-5152-42de-b718-0b01e6ae561b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:10.057762 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193c2c1-5152-42de-b718-0b01e6ae561b None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:10.058044 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193c2c1-5152-42de-b718-0b01e6ae561b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:10.058323 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193c2c1-5152-42de-b718-0b01e6ae561b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:10.059501 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d193c2c1-5152-42de-b718-0b01e6ae561b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:10.060960 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d193c2c1-5152-42de-b718-0b01e6ae561b None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:10.062645 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 125/246] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:28:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:10.063911 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf7d017-3f10-4221-8afe-dda371181bcd None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:10.064205 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf7d017-3f10-4221-8afe-dda371181bcd None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:10.064485 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf7d017-3f10-4221-8afe-dda371181bcd None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:10.066420 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 126/247] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:10.067045 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf7d017-3f10-4221-8afe-dda371181bcd None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:10.124036 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:10.132106 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 122/248] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:28:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 16:28:10.133874 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85968232-fc36-408c-ab49-7dc02ac4ffbc None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:10.134250 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85968232-fc36-408c-ab49-7dc02ac4ffbc None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:10.134539 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85968232-fc36-408c-ab49-7dc02ac4ffbc None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:10.135894 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 123/249] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:10.136589 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85968232-fc36-408c-ab49-7dc02ac4ffbc None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:10.169549 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf7d017-3f10-4221-8afe-dda371181bcd None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:10.207183 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2443ff9-172f-4fd8-a776-729c6634ab21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:10.208055 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2443ff9-172f-4fd8-a776-729c6634ab21 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:10.208346 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2443ff9-172f-4fd8-a776-729c6634ab21 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:10.208620 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2443ff9-172f-4fd8-a776-729c6634ab21 None admin] PATH_INFO: `/v3/users/7654546351bf4ed18049c396e3d0a371` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:10.209825 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2443ff9-172f-4fd8-a776-729c6634ab21 None admin] RBAC: Authorizing `identity:get_user(user_id=7654546351bf4ed18049c396e3d0a371)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:10.211275 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2443ff9-172f-4fd8-a776-729c6634ab21 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:10.212920 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 127/250] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:28:10 2022] GET /identity/v3/users/7654546351bf4ed18049c396e3d0a371 => generated 283 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:10.212920 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c180212-a5e5-4212-bddd-9da85dceccb8 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:10.212920 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c180212-a5e5-4212-bddd-9da85dceccb8 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:10.212920 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c180212-a5e5-4212-bddd-9da85dceccb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:10.230388 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-8c180212-a5e5-4212-bddd-9da85dceccb8 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:10.231570 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-8c180212-a5e5-4212-bddd-9da85dceccb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:10.239884 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85968232-fc36-408c-ab49-7dc02ac4ffbc None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:10.271060 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:10.276323 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c063759-4297-472e-b853-c07d0947809f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:10.276994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c063759-4297-472e-b853-c07d0947809f None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:10.277319 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c063759-4297-472e-b853-c07d0947809f None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:10.277600 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c063759-4297-472e-b853-c07d0947809f None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:10.278335 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c063759-4297-472e-b853-c07d0947809f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:10.279843 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c063759-4297-472e-b853-c07d0947809f None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:10.306646 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 128/251] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:28:10.306858 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 124/252] 10.222.0.36 () {60 vars in 1132 bytes} [Thu May 19 16:28:10 2022] POST /identity/v3/users => generated 293 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:28:10.306952 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c180212-a5e5-4212-bddd-9da85dceccb8 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:10.307039 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e69ac-36c9-4137-8823-49ccca537c23 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:10.307039 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e69ac-36c9-4137-8823-49ccca537c23 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:10.307039 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e69ac-36c9-4137-8823-49ccca537c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:10.325673 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-234e69ac-36c9-4137-8823-49ccca537c23 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:10.326920 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-234e69ac-36c9-4137-8823-49ccca537c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:10.387960 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:10.408220 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 125/253] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:28:10.409047 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234e69ac-36c9-4137-8823-49ccca537c23 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:10.455056 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c180212-a5e5-4212-bddd-9da85dceccb8 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:10.495749 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e5100a-0f67-45d3-95e9-899c5f3048bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:10.496734 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5100a-0f67-45d3-95e9-899c5f3048bc None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:10.497079 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5100a-0f67-45d3-95e9-899c5f3048bc None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:10.498091 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5100a-0f67-45d3-95e9-899c5f3048bc None admin] PATH_INFO: `/v3/projects/230decc93e3a483aa3d5cc33c0f662a9` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:10.498725 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e5100a-0f67-45d3-95e9-899c5f3048bc None admin] RBAC: Authorizing `identity:get_project(project_id=230decc93e3a483aa3d5cc33c0f662a9)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:10.500255 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e5100a-0f67-45d3-95e9-899c5f3048bc None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:10.502173 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 129/254] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:28:10 2022] GET /identity/v3/projects/230decc93e3a483aa3d5cc33c0f662a9 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:10.504218 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e5100a-0f67-45d3-95e9-899c5f3048bc None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:10.532999 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234e69ac-36c9-4137-8823-49ccca537c23 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:10.553787 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:10.585890 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b83345d-ab4d-480a-a030-180d4271d0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:10.586571 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b83345d-ab4d-480a-a030-180d4271d0d8 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:10.586855 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b83345d-ab4d-480a-a030-180d4271d0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:10.587134 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b83345d-ab4d-480a-a030-180d4271d0d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:10.643701 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e5100a-0f67-45d3-95e9-899c5f3048bc None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:10.665674 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:10.684086 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b83345d-ab4d-480a-a030-180d4271d0d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:10.685639 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b83345d-ab4d-480a-a030-180d4271d0d8 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:10.688144 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e680274-d8df-43b7-b7b1-501d3a861694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:10.689033 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e680274-d8df-43b7-b7b1-501d3a861694 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:10.690776 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e680274-d8df-43b7-b7b1-501d3a861694 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:10.691062 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e680274-d8df-43b7-b7b1-501d3a861694 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:10.703967 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e680274-d8df-43b7-b7b1-501d3a861694 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:28:10.712044 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-4e680274-d8df-43b7-b7b1-501d3a861694 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 16:28:10.712044 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:10.712044 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:10.712044 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:10.712044 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:10.712044 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:10.712044 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:10.712044 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:10.712044 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:10.712044 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:10.712044 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:10.712044 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:10.712044 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:28:10.712044 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:28:10.712044 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:28:10.712044 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:28:10.712985 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:28:10.712985 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:28:10.712985 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:28:10.712985 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:28:10.712985 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:10.712985 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:10.712985 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:10.712985 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:10.712985 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:10.712985 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:10.712985 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:10.712985 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:28:10.712985 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:10.712985 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:10.712985 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:10.712985 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:10.713815 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:10.713815 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:10.713815 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:10.713815 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:10.713815 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:28:10.713815 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:28:10.713815 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:28:10.713815 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:28:10.713815 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:28:10.713815 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:28:10.713815 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 16:28:10.713815 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:28:10.713815 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 130/255] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:28:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:10.714458 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e680274-d8df-43b7-b7b1-501d3a861694 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:10.804483 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 126/256] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:28:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 16:28:10.805506 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b83345d-ab4d-480a-a030-180d4271d0d8 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:10.874001 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e680274-d8df-43b7-b7b1-501d3a861694 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:10.876394 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:10.911337 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:10.918675 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd86307-b35b-46f6-9bf9-356fa0a8aea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:10.919897 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd86307-b35b-46f6-9bf9-356fa0a8aea5 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:10.920394 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd86307-b35b-46f6-9bf9-356fa0a8aea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:10.920905 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd86307-b35b-46f6-9bf9-356fa0a8aea5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:10.932481 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b83345d-ab4d-480a-a030-180d4271d0d8 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:10.939308 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd86307-b35b-46f6-9bf9-356fa0a8aea5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:28:10.947956 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-9fd86307-b35b-46f6-9bf9-356fa0a8aea5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 16:28:10.947956 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:10.947956 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:10.947956 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:10.947956 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:10.947956 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:10.947956 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:10.947956 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:10.947956 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:10.947956 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:10.947956 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:10.947956 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:10.947956 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:28:10.947956 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:28:10.947956 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:28:10.947956 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:28:10.948876 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:28:10.948876 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:28:10.948876 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:28:10.948876 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:28:10.948876 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:10.948876 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:10.948876 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:10.948876 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:10.948876 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:10.948876 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:10.948876 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:10.948876 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:28:10.948876 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:10.948876 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:10.948876 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:10.948876 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:10.950013 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:10.950013 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:10.950013 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:10.950013 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:10.950013 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:28:10.950013 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:28:10.950013 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:28:10.950013 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:28:10.950013 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:28:10.950013 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:28:10.950013 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 16:28:10.950013 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:28:10.952194 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 131/257] 10.222.0.36 () {56 vars in 1114 bytes} [Thu May 19 16:28:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:10.953301 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd86307-b35b-46f6-9bf9-356fa0a8aea5 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:10.970815 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bcf415-6f80-43e9-946c-adaec2dd99f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:10.971950 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bcf415-6f80-43e9-946c-adaec2dd99f2 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:10.972241 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bcf415-6f80-43e9-946c-adaec2dd99f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:10.972524 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bcf415-6f80-43e9-946c-adaec2dd99f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:10.973523 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bcf415-6f80-43e9-946c-adaec2dd99f2 None admin] RBAC: Adding query filter params (role.id=156d234cfe9d4844933886f1ba568694, scope.project.id=230decc93e3a483aa3d5cc33c0f662a9, user.id=24b89cd5cc53406db6fdbcbe2b60ad7b) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:10.973842 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bcf415-6f80-43e9-946c-adaec2dd99f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:10.975319 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bcf415-6f80-43e9-946c-adaec2dd99f2 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:10.989254 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 127/258] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:28:10 2022] GET /identity/v3/role_assignments?user.id=24b89cd5cc53406db6fdbcbe2b60ad7b&scope.project.id=230decc93e3a483aa3d5cc33c0f662a9&role.id=156d234cfe9d4844933886f1ba568694 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:10.990395 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bcf415-6f80-43e9-946c-adaec2dd99f2 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:11.068463 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd86307-b35b-46f6-9bf9-356fa0a8aea5 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:11.111688 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b66a0-e78d-45dc-992e-9fad04d0bf2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:11.113481 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b66a0-e78d-45dc-992e-9fad04d0bf2a None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:11.114861 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b66a0-e78d-45dc-992e-9fad04d0bf2a None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:11.115562 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b66a0-e78d-45dc-992e-9fad04d0bf2a None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:11.118084 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65b66a0-e78d-45dc-992e-9fad04d0bf2a None admin] RBAC: Adding query filter params (name=member) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:11.118710 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65b66a0-e78d-45dc-992e-9fad04d0bf2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:11.121327 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65b66a0-e78d-45dc-992e-9fad04d0bf2a None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:11.140212 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 132/259] 10.222.0.36 () {56 vars in 1101 bytes} [Thu May 19 16:28:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:11.144885 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b66a0-e78d-45dc-992e-9fad04d0bf2a None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:11.196613 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:11.205201 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bcf415-6f80-43e9-946c-adaec2dd99f2 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:11.292338 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d164f7bd-fb91-4120-849f-f55854e4f75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:11.293140 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d164f7bd-fb91-4120-849f-f55854e4f75f None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:11.294120 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d164f7bd-fb91-4120-849f-f55854e4f75f None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:11.294405 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d164f7bd-fb91-4120-849f-f55854e4f75f None admin] PATH_INFO: `/v3/projects/230decc93e3a483aa3d5cc33c0f662a9` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:11.297873 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d164f7bd-fb91-4120-849f-f55854e4f75f None admin] RBAC: Authorizing `identity:get_project(project_id=230decc93e3a483aa3d5cc33c0f662a9)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:11.299444 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d164f7bd-fb91-4120-849f-f55854e4f75f None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:11.301266 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 128/260] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:28:10 2022] GET /identity/v3/projects/230decc93e3a483aa3d5cc33c0f662a9 => generated 302 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:11.302051 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d164f7bd-fb91-4120-849f-f55854e4f75f None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:11.351794 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b66a0-e78d-45dc-992e-9fad04d0bf2a None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:11.357177 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:11.388221 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6935dcd-0bd3-45e5-9120-22dbf71556ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:11.388916 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6935dcd-0bd3-45e5-9120-22dbf71556ae None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:11.389231 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6935dcd-0bd3-45e5-9120-22dbf71556ae None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:11.389518 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6935dcd-0bd3-45e5-9120-22dbf71556ae None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:11.390283 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6935dcd-0bd3-45e5-9120-22dbf71556ae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:11.390596 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6935dcd-0bd3-45e5-9120-22dbf71556ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:11.391998 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6935dcd-0bd3-45e5-9120-22dbf71556ae None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:11.402467 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 133/261] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:28:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:11.403303 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6935dcd-0bd3-45e5-9120-22dbf71556ae None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:11.487900 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d164f7bd-fb91-4120-849f-f55854e4f75f None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:11.524663 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f335d68b-c208-4334-9b22-5197c1921599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:11.525589 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f335d68b-c208-4334-9b22-5197c1921599 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:11.525876 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f335d68b-c208-4334-9b22-5197c1921599 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:11.526155 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f335d68b-c208-4334-9b22-5197c1921599 None admin] PATH_INFO: `/v3/users/c068746943354012a817dc656ba0a7af` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:11.529990 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:11.530107 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f335d68b-c208-4334-9b22-5197c1921599 None admin] RBAC: Authorizing `identity:get_user(user_id=c068746943354012a817dc656ba0a7af)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:11.531560 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f335d68b-c208-4334-9b22-5197c1921599 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:11.536933 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 129/262] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:28:11 2022] GET /identity/v3/users/c068746943354012a817dc656ba0a7af => generated 297 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:11.539478 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f335d68b-c208-4334-9b22-5197c1921599 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:11.562118 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6935dcd-0bd3-45e5-9120-22dbf71556ae None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:11.619919 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a4b380-fc16-4756-8310-f326b15556d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:11.620594 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a4b380-fc16-4756-8310-f326b15556d1 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:11.620886 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a4b380-fc16-4756-8310-f326b15556d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:11.621184 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a4b380-fc16-4756-8310-f326b15556d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:11.622046 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a4b380-fc16-4756-8310-f326b15556d1 None admin] RBAC: Adding query filter params (role.id=e2915fe109244a07a9be96bd2b219fdd, scope.project.id=230decc93e3a483aa3d5cc33c0f662a9, user.id=7654546351bf4ed18049c396e3d0a371) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:11.622364 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a4b380-fc16-4756-8310-f326b15556d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:11.624392 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a4b380-fc16-4756-8310-f326b15556d1 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:11.635796 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 134/263] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:28:11 2022] GET /identity/v3/role_assignments?user.id=7654546351bf4ed18049c396e3d0a371&scope.project.id=230decc93e3a483aa3d5cc33c0f662a9&role.id=e2915fe109244a07a9be96bd2b219fdd => generated 608 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:11.636849 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a4b380-fc16-4756-8310-f326b15556d1 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:11.752476 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:11.793066 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f335d68b-c208-4334-9b22-5197c1921599 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:11.811416 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a4b380-fc16-4756-8310-f326b15556d1 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:11.885978 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01cb294-85a5-4678-a195-dae8aba0d088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:11.886915 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01cb294-85a5-4678-a195-dae8aba0d088 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:11.889870 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01cb294-85a5-4678-a195-dae8aba0d088 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:11.889870 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01cb294-85a5-4678-a195-dae8aba0d088 None admin] PATH_INFO: `/v3/users/7654546351bf4ed18049c396e3d0a371` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:11.889870 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01cb294-85a5-4678-a195-dae8aba0d088 None admin] RBAC: Authorizing `identity:get_user(user_id=7654546351bf4ed18049c396e3d0a371)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:11.890348 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01cb294-85a5-4678-a195-dae8aba0d088 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:11.892652 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 130/264] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:28:11 2022] GET /identity/v3/users/7654546351bf4ed18049c396e3d0a371 => generated 283 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:11.898045 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee0f89-9982-46c1-a0e2-89e710e78df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:11.898777 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01cb294-85a5-4678-a195-dae8aba0d088 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:11.899852 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee0f89-9982-46c1-a0e2-89e710e78df6 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:11.900143 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee0f89-9982-46c1-a0e2-89e710e78df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:11.900426 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee0f89-9982-46c1-a0e2-89e710e78df6 None admin] PATH_INFO: `/v3/projects/230decc93e3a483aa3d5cc33c0f662a9` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:11.906018 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbee0f89-9982-46c1-a0e2-89e710e78df6 None admin] RBAC: Authorizing `identity:get_project(project_id=230decc93e3a483aa3d5cc33c0f662a9)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:11.907539 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbee0f89-9982-46c1-a0e2-89e710e78df6 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:11.910024 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 135/265] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:28:11 2022] GET /identity/v3/projects/230decc93e3a483aa3d5cc33c0f662a9 => generated 302 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:11.916946 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee0f89-9982-46c1-a0e2-89e710e78df6 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:11.969625 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:12.016195 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01cb294-85a5-4678-a195-dae8aba0d088 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:12.058174 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2b4799-3ea7-47ae-8e48-9eabb6f956fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:12.059275 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b4799-3ea7-47ae-8e48-9eabb6f956fb None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:12.059825 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b4799-3ea7-47ae-8e48-9eabb6f956fb None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:12.060340 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b4799-3ea7-47ae-8e48-9eabb6f956fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:12.088888 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:12.097332 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee0f89-9982-46c1-a0e2-89e710e78df6 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:12.138410 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2b4799-3ea7-47ae-8e48-9eabb6f956fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:12.140377 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2b4799-3ea7-47ae-8e48-9eabb6f956fb None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:12.149429 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3b8f52-520a-481c-9dd9-6bf04b6da43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:12.150365 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b8f52-520a-481c-9dd9-6bf04b6da43a None admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:12.150629 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b8f52-520a-481c-9dd9-6bf04b6da43a None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:12.150921 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b8f52-520a-481c-9dd9-6bf04b6da43a None admin] PATH_INFO: `/v3/projects/230decc93e3a483aa3d5cc33c0f662a9/users/c068746943354012a817dc656ba0a7af/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:12.153005 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3b8f52-520a-481c-9dd9-6bf04b6da43a None admin] RBAC: Authorizing `identity:create_grant(project_id=230decc93e3a483aa3d5cc33c0f662a9, user_id=c068746943354012a817dc656ba0a7af, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:12.158485 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3b8f52-520a-481c-9dd9-6bf04b6da43a None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:12.179872 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 136/266] 10.222.0.36 () {58 vars in 1439 bytes} [Thu May 19 16:28:11 2022] PUT /identity/v3/projects/230decc93e3a483aa3d5cc33c0f662a9/users/c068746943354012a817dc656ba0a7af/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:28:12.260186 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 131/267] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:28:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 16:28:12.275607 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3b8f52-520a-481c-9dd9-6bf04b6da43a None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:12.396598 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:12.424743 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3b8f52-520a-481c-9dd9-6bf04b6da43a None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:12.466306 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f79cc7-9adb-4486-a73c-118b506599ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:12.467183 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f79cc7-9adb-4486-a73c-118b506599ff None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:12.467472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f79cc7-9adb-4486-a73c-118b506599ff None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:12.467758 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f79cc7-9adb-4486-a73c-118b506599ff None admin] PATH_INFO: `/v3/projects/230decc93e3a483aa3d5cc33c0f662a9` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:12.468976 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f79cc7-9adb-4486-a73c-118b506599ff None admin] RBAC: Authorizing `identity:get_project(project_id=230decc93e3a483aa3d5cc33c0f662a9)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:12.470473 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f79cc7-9adb-4486-a73c-118b506599ff None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:12.472232 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 137/268] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:28:12 2022] GET /identity/v3/projects/230decc93e3a483aa3d5cc33c0f662a9 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:12.478196 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2b4799-3ea7-47ae-8e48-9eabb6f956fb None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:12.539356 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:12.597704 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2b4799-3ea7-47ae-8e48-9eabb6f956fb None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:12.652319 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c420db6-c164-4275-aaae-6de3df345c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:12.653348 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c420db6-c164-4275-aaae-6de3df345c56 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:12.653841 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c420db6-c164-4275-aaae-6de3df345c56 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:12.654342 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c420db6-c164-4275-aaae-6de3df345c56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:12.655392 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c420db6-c164-4275-aaae-6de3df345c56 None admin] RBAC: Adding query filter params (role.id=50f62a33eb434682baa7529619fe14b0, scope.project.id=230decc93e3a483aa3d5cc33c0f662a9, user.id=7654546351bf4ed18049c396e3d0a371) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:12.655911 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c420db6-c164-4275-aaae-6de3df345c56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:12.657603 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c420db6-c164-4275-aaae-6de3df345c56 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:12.670185 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 132/269] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:28:12 2022] GET /identity/v3/role_assignments?user.id=7654546351bf4ed18049c396e3d0a371&scope.project.id=230decc93e3a483aa3d5cc33c0f662a9&role.id=50f62a33eb434682baa7529619fe14b0 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:13.008154 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14618654-86b6-40d7-a0b1-a1704452c91d None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:13.009836 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14618654-86b6-40d7-a0b1-a1704452c91d None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:13.009836 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14618654-86b6-40d7-a0b1-a1704452c91d None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:13.013180 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 138/270] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:13.017107 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb16e9-ff10-4cfe-a9c6-24fa7676ee77 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:13.017660 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb16e9-ff10-4cfe-a9c6-24fa7676ee77 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:13.017739 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb16e9-ff10-4cfe-a9c6-24fa7676ee77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:13.037015 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-cecb16e9-ff10-4cfe-a9c6-24fa7676ee77 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:13.038325 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-cecb16e9-ff10-4cfe-a9c6-24fa7676ee77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:13.110601 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 133/271] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:28:13.134488 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14618654-86b6-40d7-a0b1-a1704452c91d None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:13.227108 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4effa17-b641-4279-b6fb-3616d9e65e91 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:13.227422 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4effa17-b641-4279-b6fb-3616d9e65e91 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:13.227677 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4effa17-b641-4279-b6fb-3616d9e65e91 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:13.229186 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 134/272] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 16:28:13.234802 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad09a4-b60e-4850-9a36-9e32d6fffded None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:13.235101 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad09a4-b60e-4850-9a36-9e32d6fffded None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:13.235384 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad09a4-b60e-4850-9a36-9e32d6fffded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:13.260613 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14618654-86b6-40d7-a0b1-a1704452c91d None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:13.261852 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-33ad09a4-b60e-4850-9a36-9e32d6fffded None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:13.263470 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-33ad09a4-b60e-4850-9a36-9e32d6fffded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:13.304380 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e2c16e-ef2c-4c48-810b-16dcdf1f0537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:13.305212 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2c16e-ef2c-4c48-810b-16dcdf1f0537 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:13.305513 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2c16e-ef2c-4c48-810b-16dcdf1f0537 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:13.305970 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2c16e-ef2c-4c48-810b-16dcdf1f0537 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:13.319468 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:13.320372 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e2c16e-ef2c-4c48-810b-16dcdf1f0537 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:28:13.331383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-96e2c16e-ef2c-4c48-810b-16dcdf1f0537 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 16:28:13.331383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:13.331383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:13.331383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:13.331383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:13.331383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:13.331383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:13.331383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:13.331383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:13.331383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:13.331383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:13.331383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:13.331383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:28:13.331383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:28:13.331383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:28:13.331383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:28:13.332299 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:28:13.332299 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:28:13.332299 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:28:13.332299 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:28:13.332299 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:13.332299 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:13.332299 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:13.332299 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:13.332299 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:13.332299 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:13.332299 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:13.332299 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:28:13.332299 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:13.332299 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:13.332299 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:13.332299 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:13.333091 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:13.333091 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:13.333091 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:13.333091 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:13.333091 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:28:13.333091 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:28:13.333091 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:28:13.333091 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:28:13.333091 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:28:13.333091 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:28:13.333091 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 16:28:13.333091 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:28:13.337231 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 139/273] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:28:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:13.339555 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e2c16e-ef2c-4c48-810b-16dcdf1f0537 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:13.350008 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:13.354311 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 135/274] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:28:13.379061 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad09a4-b60e-4850-9a36-9e32d6fffded None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:13.448855 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e2c16e-ef2c-4c48-810b-16dcdf1f0537 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:13.484752 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e76c9c7-7d43-4eba-b152-b723fb522fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:13.485614 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76c9c7-7d43-4eba-b152-b723fb522fc7 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:13.485905 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76c9c7-7d43-4eba-b152-b723fb522fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:13.486507 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76c9c7-7d43-4eba-b152-b723fb522fc7 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:13.488098 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e76c9c7-7d43-4eba-b152-b723fb522fc7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:13.488415 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e76c9c7-7d43-4eba-b152-b723fb522fc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:13.490315 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e76c9c7-7d43-4eba-b152-b723fb522fc7 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:13.492931 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad09a4-b60e-4850-9a36-9e32d6fffded None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:13.500678 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 140/275] 10.222.0.36 () {56 vars in 1101 bytes} [Thu May 19 16:28:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:28:13.506758 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e76c9c7-7d43-4eba-b152-b723fb522fc7 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:13.532849 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4775f48-1ce2-41da-b0bc-30060f8e0943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:13.533780 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4775f48-1ce2-41da-b0bc-30060f8e0943 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:13.534071 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4775f48-1ce2-41da-b0bc-30060f8e0943 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:13.534351 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4775f48-1ce2-41da-b0bc-30060f8e0943 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:13.535736 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4775f48-1ce2-41da-b0bc-30060f8e0943 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:13.537392 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4775f48-1ce2-41da-b0bc-30060f8e0943 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:13.539571 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 136/276] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:28:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:28:13.544680 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4775f48-1ce2-41da-b0bc-30060f8e0943 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:13.577467 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:13.602363 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:13.614951 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e76c9c7-7d43-4eba-b152-b723fb522fc7 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:13.652131 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c32fab7-8fe0-426a-99ab-e24821b50cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:13.653041 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32fab7-8fe0-426a-99ab-e24821b50cab None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:13.653761 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32fab7-8fe0-426a-99ab-e24821b50cab None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:13.653761 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32fab7-8fe0-426a-99ab-e24821b50cab None admin] PATH_INFO: `/v3/users/46a4ef31fc9a49b7a971148550fab7dd` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:13.668399 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c32fab7-8fe0-426a-99ab-e24821b50cab None admin] RBAC: Authorizing `identity:get_user(user_id=46a4ef31fc9a49b7a971148550fab7dd)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:13.669367 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4775f48-1ce2-41da-b0bc-30060f8e0943 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:13.670006 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c32fab7-8fe0-426a-99ab-e24821b50cab None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:13.671908 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 141/277] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:28:13 2022] GET /identity/v3/users/46a4ef31fc9a49b7a971148550fab7dd => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:28:13.677975 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c32fab7-8fe0-426a-99ab-e24821b50cab None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:13.710000 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ecdf43-0a7f-41c7-aabc-da65628ac6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:13.711045 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ecdf43-0a7f-41c7-aabc-da65628ac6c5 None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:13.711558 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ecdf43-0a7f-41c7-aabc-da65628ac6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:13.712089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ecdf43-0a7f-41c7-aabc-da65628ac6c5 None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:13.713196 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ecdf43-0a7f-41c7-aabc-da65628ac6c5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:13.714902 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ecdf43-0a7f-41c7-aabc-da65628ac6c5 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:13.748562 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 137/278] 10.222.0.36 () {60 vars in 1132 bytes} [Thu May 19 16:28:13 2022] POST /identity/v3/users => generated 293 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:28:13.803143 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c32fab7-8fe0-426a-99ab-e24821b50cab None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:13.831600 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:13.842543 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a881ae-4ac6-4b21-942f-792917d9456c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:13.843219 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a881ae-4ac6-4b21-942f-792917d9456c None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:13.843506 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a881ae-4ac6-4b21-942f-792917d9456c None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:13.843787 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a881ae-4ac6-4b21-942f-792917d9456c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:13.844698 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a881ae-4ac6-4b21-942f-792917d9456c None admin] RBAC: Adding query filter params (role.id=0e98fd94a6d24e46b9fe09a22a7df04a, user.id=46a4ef31fc9a49b7a971148550fab7dd, scope.system=all) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:13.845023 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a881ae-4ac6-4b21-942f-792917d9456c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:13.847090 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a881ae-4ac6-4b21-942f-792917d9456c None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:13.857397 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 142/279] 10.222.0.36 () {56 vars in 1308 bytes} [Thu May 19 16:28:13 2022] GET /identity/v3/role_assignments?user.id=46a4ef31fc9a49b7a971148550fab7dd&scope.system=all&role.id=0e98fd94a6d24e46b9fe09a22a7df04a => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:13.941911 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642d2d2-e950-46be-9fc2-06f8ef7c2f5e None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:13.942234 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642d2d2-e950-46be-9fc2-06f8ef7c2f5e None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:13.942526 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642d2d2-e950-46be-9fc2-06f8ef7c2f5e None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:13.944363 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 138/280] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:13.951426 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8073eb4-cc04-4506-8d46-b4bfa60ac625 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:13.951798 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8073eb4-cc04-4506-8d46-b4bfa60ac625 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:13.951999 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8073eb4-cc04-4506-8d46-b4bfa60ac625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:13.974112 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-d8073eb4-cc04-4506-8d46-b4bfa60ac625 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:13.974425 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d8073eb4-cc04-4506-8d46-b4bfa60ac625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:14.057706 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 143/281] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:28:14.078175 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d642d2d2-e950-46be-9fc2-06f8ef7c2f5e None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:14.198606 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d642d2d2-e950-46be-9fc2-06f8ef7c2f5e None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:14.200244 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:14.240497 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d55a5de-1a7c-403b-910a-9d6c51c75f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:14.241372 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55a5de-1a7c-403b-910a-9d6c51c75f51 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:14.241721 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55a5de-1a7c-403b-910a-9d6c51c75f51 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:14.242007 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55a5de-1a7c-403b-910a-9d6c51c75f51 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:14.253498 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d55a5de-1a7c-403b-910a-9d6c51c75f51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:28:14.261659 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-0d55a5de-1a7c-403b-910a-9d6c51c75f51 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 16:28:14.261659 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:14.261659 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:14.261659 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:14.261659 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:14.261659 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:14.261659 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:14.261659 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:14.261659 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:14.261659 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:14.261659 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:14.261659 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:14.261659 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:28:14.261659 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:28:14.261659 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:28:14.261659 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:28:14.262558 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:28:14.262558 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:28:14.262558 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:28:14.262558 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:28:14.262558 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:14.262558 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:14.262558 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:14.262558 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:14.262558 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:14.262558 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:14.262558 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:14.262558 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:14.262558 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:14.262558 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:14.262558 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:14.262558 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:14.263350 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:14.263350 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:14.263350 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:14.263350 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:14.263350 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:28:14.263350 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:28:14.263350 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:28:14.263350 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:28:14.263350 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:28:14.263350 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:28:14.263350 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 16:28:14.263350 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:14.265340 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 139/282] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:28:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:14.269780 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8073eb4-cc04-4506-8d46-b4bfa60ac625 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:14.430063 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8073eb4-cc04-4506-8d46-b4bfa60ac625 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:14.451297 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:14.468262 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40f6fd9-2b37-4740-8436-f41bbeb4b132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:14.468979 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f6fd9-2b37-4740-8436-f41bbeb4b132 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:14.469374 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f6fd9-2b37-4740-8436-f41bbeb4b132 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:14.469661 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f6fd9-2b37-4740-8436-f41bbeb4b132 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:14.470531 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40f6fd9-2b37-4740-8436-f41bbeb4b132 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:14.470845 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40f6fd9-2b37-4740-8436-f41bbeb4b132 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:14.472951 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40f6fd9-2b37-4740-8436-f41bbeb4b132 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:14.486861 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 144/283] 10.222.0.36 () {56 vars in 1101 bytes} [Thu May 19 16:28:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:14.494321 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d55a5de-1a7c-403b-910a-9d6c51c75f51 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:14.650788 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d55a5de-1a7c-403b-910a-9d6c51c75f51 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:14.702004 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8bb40f-57a4-41c9-8b12-ebbaf33e5a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:14.702829 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8bb40f-57a4-41c9-8b12-ebbaf33e5a1d None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:14.703117 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8bb40f-57a4-41c9-8b12-ebbaf33e5a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:14.703398 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8bb40f-57a4-41c9-8b12-ebbaf33e5a1d None admin] PATH_INFO: `/v3/users/24b89cd5cc53406db6fdbcbe2b60ad7b` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:14.704748 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8bb40f-57a4-41c9-8b12-ebbaf33e5a1d None admin] RBAC: Authorizing `identity:get_user(user_id=24b89cd5cc53406db6fdbcbe2b60ad7b)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:14.706906 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8bb40f-57a4-41c9-8b12-ebbaf33e5a1d None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:14.708871 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 140/284] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:28:14 2022] GET /identity/v3/users/24b89cd5cc53406db6fdbcbe2b60ad7b => generated 297 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:14.714226 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40f6fd9-2b37-4740-8436-f41bbeb4b132 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:14.849446 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40f6fd9-2b37-4740-8436-f41bbeb4b132 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:14.885109 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb3934-f47d-4d64-a1f5-eb9383ae7479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:14.886026 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb3934-f47d-4d64-a1f5-eb9383ae7479 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:14.886316 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb3934-f47d-4d64-a1f5-eb9383ae7479 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:14.886598 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb3934-f47d-4d64-a1f5-eb9383ae7479 None admin] PATH_INFO: `/v3/projects/230decc93e3a483aa3d5cc33c0f662a9` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:14.887827 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fb3934-f47d-4d64-a1f5-eb9383ae7479 None admin] RBAC: Authorizing `identity:get_project(project_id=230decc93e3a483aa3d5cc33c0f662a9)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:14.889338 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fb3934-f47d-4d64-a1f5-eb9383ae7479 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:14.895834 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 145/285] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:28:14 2022] GET /identity/v3/projects/230decc93e3a483aa3d5cc33c0f662a9 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:14.897476 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8bb40f-57a4-41c9-8b12-ebbaf33e5a1d None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:14.940747 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:14.977480 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d35c1-e97a-426d-8306-1ac06d3d20e3 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:14.977768 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d35c1-e97a-426d-8306-1ac06d3d20e3 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:14.977978 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d35c1-e97a-426d-8306-1ac06d3d20e3 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:14.979789 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 146/286] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:14.988176 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e23971-4bcc-434d-877f-06d7d4183bbb None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:14.988467 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e23971-4bcc-434d-877f-06d7d4183bbb None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:14.988752 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e23971-4bcc-434d-877f-06d7d4183bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:15.020438 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8bb40f-57a4-41c9-8b12-ebbaf33e5a1d None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:15.022277 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-47e23971-4bcc-434d-877f-06d7d4183bbb None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:15.023621 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-47e23971-4bcc-434d-877f-06d7d4183bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:15.076020 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d0b5aa-5c5a-464b-a3d3-62af1a2b649c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:15.076919 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0b5aa-5c5a-464b-a3d3-62af1a2b649c None admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:15.077459 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0b5aa-5c5a-464b-a3d3-62af1a2b649c None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:15.078332 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0b5aa-5c5a-464b-a3d3-62af1a2b649c None admin] PATH_INFO: `/v3/projects/230decc93e3a483aa3d5cc33c0f662a9/users/24b89cd5cc53406db6fdbcbe2b60ad7b/roles/156d234cfe9d4844933886f1ba568694` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:15.081595 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d0b5aa-5c5a-464b-a3d3-62af1a2b649c None admin] RBAC: Authorizing `identity:create_grant(project_id=230decc93e3a483aa3d5cc33c0f662a9, user_id=24b89cd5cc53406db6fdbcbe2b60ad7b, role_id=156d234cfe9d4844933886f1ba568694)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:15.083128 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d0b5aa-5c5a-464b-a3d3-62af1a2b649c None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:15.107884 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 141/287] 10.222.0.36 () {58 vars in 1439 bytes} [Thu May 19 16:28:14 2022] PUT /identity/v3/projects/230decc93e3a483aa3d5cc33c0f662a9/users/24b89cd5cc53406db6fdbcbe2b60ad7b/roles/156d234cfe9d4844933886f1ba568694 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:28:15.119183 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:15.142722 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 147/288] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 16:28:15.179786 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d0b5aa-5c5a-464b-a3d3-62af1a2b649c None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:15.304022 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:15.315840 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d0b5aa-5c5a-464b-a3d3-62af1a2b649c None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:15.357545 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270fc03-ecfb-4705-835c-7b904b895eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:15.358375 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270fc03-ecfb-4705-835c-7b904b895eeb None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:15.358653 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270fc03-ecfb-4705-835c-7b904b895eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:15.358933 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270fc03-ecfb-4705-835c-7b904b895eeb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:15.379461 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3270fc03-ecfb-4705-835c-7b904b895eeb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:28:15.387408 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-3270fc03-ecfb-4705-835c-7b904b895eeb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 16:28:15.387408 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:15.387408 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:15.387408 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:15.387408 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:15.387408 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:15.387408 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:15.387408 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:15.387408 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:15.387408 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:15.387408 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:15.387408 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:15.387408 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:28:15.387408 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:28:15.387408 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:28:15.387408 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:28:15.388346 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:28:15.388346 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:28:15.388346 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:28:15.388346 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:28:15.388346 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:15.388346 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:15.388346 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:15.388346 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:15.388346 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:15.388346 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:15.388346 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:15.388346 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:15.388346 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:15.388346 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:15.388346 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:15.388346 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:15.389616 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:15.389616 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:15.389616 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:15.389616 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:15.389616 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:28:15.389616 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:28:15.389616 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:28:15.389616 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:28:15.389616 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:28:15.389616 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:28:15.389616 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 16:28:15.389616 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:15.389616 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 142/289] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:28:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:15.398518 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e23971-4bcc-434d-877f-06d7d4183bbb None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:15.524547 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e23971-4bcc-434d-877f-06d7d4183bbb None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:15.562911 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669983e-5267-41da-8047-94cff2044d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:15.563537 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669983e-5267-41da-8047-94cff2044d1c None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:15.563826 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669983e-5267-41da-8047-94cff2044d1c None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:15.564116 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669983e-5267-41da-8047-94cff2044d1c None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:15.564961 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669983e-5267-41da-8047-94cff2044d1c None admin] RBAC: Adding query filter params (name=member) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:15.565317 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669983e-5267-41da-8047-94cff2044d1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:15.566960 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669983e-5267-41da-8047-94cff2044d1c None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:15.577127 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 148/290] 10.222.0.36 () {56 vars in 1101 bytes} [Thu May 19 16:28:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:15.582605 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270fc03-ecfb-4705-835c-7b904b895eeb None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:15.718123 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270fc03-ecfb-4705-835c-7b904b895eeb None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:15.753737 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deaa917-9b9a-4d71-8fbf-8b509785c903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:15.755003 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deaa917-9b9a-4d71-8fbf-8b509785c903 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:15.755482 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deaa917-9b9a-4d71-8fbf-8b509785c903 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:15.755963 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deaa917-9b9a-4d71-8fbf-8b509785c903 None admin] PATH_INFO: `/v3/users/c068746943354012a817dc656ba0a7af` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:15.757452 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deaa917-9b9a-4d71-8fbf-8b509785c903 None admin] RBAC: Authorizing `identity:get_user(user_id=c068746943354012a817dc656ba0a7af)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:15.759208 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deaa917-9b9a-4d71-8fbf-8b509785c903 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:15.761051 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 143/291] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:28:15 2022] GET /identity/v3/users/c068746943354012a817dc656ba0a7af => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:15.766605 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669983e-5267-41da-8047-94cff2044d1c None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:15.837552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:15.898951 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669983e-5267-41da-8047-94cff2044d1c None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:15.934234 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e162bae1-6e83-45ed-b19a-37c1ac5fbf2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:15.934905 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e162bae1-6e83-45ed-b19a-37c1ac5fbf2f None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:15.935195 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e162bae1-6e83-45ed-b19a-37c1ac5fbf2f None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:15.935484 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e162bae1-6e83-45ed-b19a-37c1ac5fbf2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:16.008412 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e162bae1-6e83-45ed-b19a-37c1ac5fbf2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:16.009960 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e162bae1-6e83-45ed-b19a-37c1ac5fbf2f None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:16.057410 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:16.118951 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 149/292] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:28:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 16:28:16.124155 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deaa917-9b9a-4d71-8fbf-8b509785c903 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:16.162676 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:16.258874 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deaa917-9b9a-4d71-8fbf-8b509785c903 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:16.300799 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5382068d-743b-4b32-9945-77f1c9f50a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:16.301985 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5382068d-743b-4b32-9945-77f1c9f50a7d None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:16.302413 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5382068d-743b-4b32-9945-77f1c9f50a7d None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:16.302832 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5382068d-743b-4b32-9945-77f1c9f50a7d None admin] PATH_INFO: `/v3/projects/230decc93e3a483aa3d5cc33c0f662a9` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:16.304220 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5382068d-743b-4b32-9945-77f1c9f50a7d None admin] RBAC: Authorizing `identity:get_project(project_id=230decc93e3a483aa3d5cc33c0f662a9)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:16.305943 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5382068d-743b-4b32-9945-77f1c9f50a7d None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:16.308810 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 144/293] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:28:16 2022] GET /identity/v3/projects/230decc93e3a483aa3d5cc33c0f662a9 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:16.313733 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e162bae1-6e83-45ed-b19a-37c1ac5fbf2f None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:16.476705 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e162bae1-6e83-45ed-b19a-37c1ac5fbf2f None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:16.527172 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb34649-dc3d-4ee4-b263-9931801ccf69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:16.527835 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb34649-dc3d-4ee4-b263-9931801ccf69 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:16.528121 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb34649-dc3d-4ee4-b263-9931801ccf69 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:16.528399 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb34649-dc3d-4ee4-b263-9931801ccf69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:16.529278 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb34649-dc3d-4ee4-b263-9931801ccf69 None admin] RBAC: Adding query filter params (role.id=0e98fd94a6d24e46b9fe09a22a7df04a, scope.project.id=230decc93e3a483aa3d5cc33c0f662a9, user.id=c068746943354012a817dc656ba0a7af) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:16.529595 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb34649-dc3d-4ee4-b263-9931801ccf69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:16.531057 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb34649-dc3d-4ee4-b263-9931801ccf69 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:16.540057 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 150/294] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:28:16 2022] GET /identity/v3/role_assignments?user.id=c068746943354012a817dc656ba0a7af&scope.project.id=230decc93e3a483aa3d5cc33c0f662a9&role.id=0e98fd94a6d24e46b9fe09a22a7df04a => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:16.666771 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18808fa2-bb38-4e7f-ac16-2ddc058c95ce None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:16.667417 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18808fa2-bb38-4e7f-ac16-2ddc058c95ce None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:16.667899 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18808fa2-bb38-4e7f-ac16-2ddc058c95ce None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:16.669991 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 145/295] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:16.676796 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7962e-b595-4c80-a7c9-02082e0dc692 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:16.677518 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7962e-b595-4c80-a7c9-02082e0dc692 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:16.678007 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7962e-b595-4c80-a7c9-02082e0dc692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:16.692910 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:16.707652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-dfc7962e-b595-4c80-a7c9-02082e0dc692 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:16.709312 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-dfc7962e-b595-4c80-a7c9-02082e0dc692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:16.804266 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 151/296] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 16:28:16.840148 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18808fa2-bb38-4e7f-ac16-2ddc058c95ce None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:16.860086 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea959913-4186-4218-9b95-cd0802a58a6c None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:16.860391 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea959913-4186-4218-9b95-cd0802a58a6c None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:16.860645 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea959913-4186-4218-9b95-cd0802a58a6c None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:16.862959 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 152/297] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:16.870031 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e78ca-58ec-4692-9405-ff618668afc2 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:16.870341 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e78ca-58ec-4692-9405-ff618668afc2 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:16.870592 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e78ca-58ec-4692-9405-ff618668afc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:16.894706 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-518e78ca-58ec-4692-9405-ff618668afc2 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:16.896228 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-518e78ca-58ec-4692-9405-ff618668afc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:16.962042 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:16.982444 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18808fa2-bb38-4e7f-ac16-2ddc058c95ce None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:16.985248 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 153/298] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 16:28:16.987396 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1750e0-1602-40df-9e80-9c041d375c5b None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:16.987932 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1750e0-1602-40df-9e80-9c041d375c5b None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:16.988414 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1750e0-1602-40df-9e80-9c041d375c5b None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:16.989992 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 154/299] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:16.995451 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1aa88c-85a2-41aa-b2a2-42362be7a839 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:16.995979 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1aa88c-85a2-41aa-b2a2-42362be7a839 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:16.996482 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1aa88c-85a2-41aa-b2a2-42362be7a839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:17.015962 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-7b1aa88c-85a2-41aa-b2a2-42362be7a839 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:17.017588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-7b1aa88c-85a2-41aa-b2a2-42362be7a839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:17.020376 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1b4412-cfbb-4cf4-ae63-2774b594964f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:17.021602 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b4412-cfbb-4cf4-ae63-2774b594964f None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:17.022065 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b4412-cfbb-4cf4-ae63-2774b594964f None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:17.022799 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b4412-cfbb-4cf4-ae63-2774b594964f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:17.024903 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1b4412-cfbb-4cf4-ae63-2774b594964f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:17.026753 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1b4412-cfbb-4cf4-ae63-2774b594964f None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:17.028752 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 146/300] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:28:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:17.035053 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1b4412-cfbb-4cf4-ae63-2774b594964f None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:17.084702 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:17.094063 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 155/301] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 16:28:17.094758 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1aa88c-85a2-41aa-b2a2-42362be7a839 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:17.158165 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1b4412-cfbb-4cf4-ae63-2774b594964f None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:17.197120 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf30660-027f-4fbe-84c3-d65851ae6864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:17.198015 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf30660-027f-4fbe-84c3-d65851ae6864 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:17.198308 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf30660-027f-4fbe-84c3-d65851ae6864 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:17.198742 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf30660-027f-4fbe-84c3-d65851ae6864 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:17.201541 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1aa88c-85a2-41aa-b2a2-42362be7a839 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:17.209806 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf30660-027f-4fbe-84c3-d65851ae6864 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:28:17.217502 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-1bf30660-027f-4fbe-84c3-d65851ae6864 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 16:28:17.217502 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:17.217502 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:17.217502 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:17.217502 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:17.217502 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:17.217502 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:17.217502 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:17.217502 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:17.217502 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:17.217502 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:17.217502 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:17.217502 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:28:17.217502 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:28:17.217502 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:28:17.217502 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:28:17.218425 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:28:17.218425 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:28:17.218425 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:28:17.218425 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:28:17.218425 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:17.218425 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:17.218425 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:17.218425 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:17.218425 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:17.218425 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:17.218425 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:17.218425 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:17.218425 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:17.218425 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:17.218425 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:17.218425 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:17.219268 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:17.219268 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:17.219268 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:17.219268 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:17.219268 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:28:17.219268 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:28:17.219268 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:28:17.219268 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:28:17.219268 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:28:17.219268 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:28:17.219268 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 16:28:17.219268 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:17.219268 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 147/302] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:28:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:17.220055 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf30660-027f-4fbe-84c3-d65851ae6864 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:17.238463 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:17.245139 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8c9fe3-3b96-41f1-9738-cf82935c3372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:17.245849 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c9fe3-3b96-41f1-9738-cf82935c3372 None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:17.246126 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c9fe3-3b96-41f1-9738-cf82935c3372 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:17.246405 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c9fe3-3b96-41f1-9738-cf82935c3372 None admin] PATH_INFO: `/v3/groups` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:17.247096 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8c9fe3-3b96-41f1-9738-cf82935c3372 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:17.248561 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8c9fe3-3b96-41f1-9738-cf82935c3372 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:17.264295 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 156/303] 10.222.0.36 () {60 vars in 1134 bytes} [Thu May 19 16:28:17 2022] POST /identity/v3/groups => generated 229 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:28:17.271354 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8c9fe3-3b96-41f1-9738-cf82935c3372 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:17.346776 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf30660-027f-4fbe-84c3-d65851ae6864 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:17.364874 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:17.399136 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e0bc7-afd5-4b95-afda-173f04c14c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:17.400297 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e0bc7-afd5-4b95-afda-173f04c14c93 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:17.401231 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e0bc7-afd5-4b95-afda-173f04c14c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:17.401668 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e0bc7-afd5-4b95-afda-173f04c14c93 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:17.421325 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582e0bc7-afd5-4b95-afda-173f04c14c93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:28:17.429663 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8c9fe3-3b96-41f1-9738-cf82935c3372 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:17.433577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-582e0bc7-afd5-4b95-afda-173f04c14c93 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 16:28:17.433577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:17.433577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:17.433577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:17.433577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:17.433577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:17.433577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:17.433577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:17.433577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:17.433577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:17.433577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:17.433577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:17.433577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:28:17.433577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:28:17.433577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:28:17.433577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:28:17.434534 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:28:17.434534 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:28:17.434534 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:28:17.434534 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:28:17.434534 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:17.434534 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:17.434534 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:17.434534 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:17.434534 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:17.434534 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:17.434534 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:17.434534 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:17.434534 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:17.434534 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:17.434534 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:17.434534 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:17.435356 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:17.435356 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:17.435356 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:17.435356 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:17.435356 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:28:17.435356 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:28:17.435356 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:28:17.435356 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:28:17.435356 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:28:17.435356 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:28:17.435356 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 16:28:17.435356 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:17.439288 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 148/304] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:28:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:17.445336 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e0bc7-afd5-4b95-afda-173f04c14c93 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:17.486880 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11623e3-0bde-4f6d-a04f-6dfb3cdf4b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:17.487575 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11623e3-0bde-4f6d-a04f-6dfb3cdf4b2b None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:17.487867 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11623e3-0bde-4f6d-a04f-6dfb3cdf4b2b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:17.488157 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11623e3-0bde-4f6d-a04f-6dfb3cdf4b2b None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:17.488926 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11623e3-0bde-4f6d-a04f-6dfb3cdf4b2b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:17.489268 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11623e3-0bde-4f6d-a04f-6dfb3cdf4b2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:17.490688 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11623e3-0bde-4f6d-a04f-6dfb3cdf4b2b None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:17.505483 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 157/305] 10.222.0.36 () {56 vars in 1101 bytes} [Thu May 19 16:28:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:17.510632 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11623e3-0bde-4f6d-a04f-6dfb3cdf4b2b None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:17.544119 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:17.573051 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e0bc7-afd5-4b95-afda-173f04c14c93 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:17.617889 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5469c2cc-3f9d-4978-9c02-b1b6796699e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:17.618762 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469c2cc-3f9d-4978-9c02-b1b6796699e6 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:17.619053 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469c2cc-3f9d-4978-9c02-b1b6796699e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:17.619334 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469c2cc-3f9d-4978-9c02-b1b6796699e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:17.620101 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5469c2cc-3f9d-4978-9c02-b1b6796699e6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:17.620646 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5469c2cc-3f9d-4978-9c02-b1b6796699e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:17.622663 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5469c2cc-3f9d-4978-9c02-b1b6796699e6 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:17.629001 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11623e3-0bde-4f6d-a04f-6dfb3cdf4b2b None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:17.635046 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 149/306] 10.222.0.36 () {56 vars in 1101 bytes} [Thu May 19 16:28:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:28:17.641142 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5469c2cc-3f9d-4978-9c02-b1b6796699e6 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:17.654035 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:17.675338 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282a6be5-6844-4926-baf3-8a3f8650a450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:17.676221 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a6be5-6844-4926-baf3-8a3f8650a450 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:17.676501 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a6be5-6844-4926-baf3-8a3f8650a450 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:17.676775 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a6be5-6844-4926-baf3-8a3f8650a450 None admin] PATH_INFO: `/v3/users/66e7ee5ae73e4723b79b9810ef646d0d` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:17.693551 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282a6be5-6844-4926-baf3-8a3f8650a450 None admin] RBAC: Authorizing `identity:get_user(user_id=66e7ee5ae73e4723b79b9810ef646d0d)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:17.698547 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282a6be5-6844-4926-baf3-8a3f8650a450 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:17.701448 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 158/307] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:28:17 2022] GET /identity/v3/users/66e7ee5ae73e4723b79b9810ef646d0d => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:17.709068 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282a6be5-6844-4926-baf3-8a3f8650a450 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:17.791894 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5469c2cc-3f9d-4978-9c02-b1b6796699e6 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:17.841347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282a6be5-6844-4926-baf3-8a3f8650a450 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:17.844635 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af449be1-7594-486e-9767-6b0682aebecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:17.846051 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af449be1-7594-486e-9767-6b0682aebecd None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:17.846051 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af449be1-7594-486e-9767-6b0682aebecd None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:17.846405 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af449be1-7594-486e-9767-6b0682aebecd None admin] PATH_INFO: `/v3/users/46a4ef31fc9a49b7a971148550fab7dd` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:17.847833 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af449be1-7594-486e-9767-6b0682aebecd None admin] RBAC: Authorizing `identity:get_user(user_id=46a4ef31fc9a49b7a971148550fab7dd)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:17.849471 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af449be1-7594-486e-9767-6b0682aebecd None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:17.851657 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 150/308] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:28:17 2022] GET /identity/v3/users/46a4ef31fc9a49b7a971148550fab7dd => generated 293 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:17.857004 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af449be1-7594-486e-9767-6b0682aebecd None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:17.862168 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:17.890963 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e512e2c3-08fa-4cec-a878-5615c666f08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:17.892323 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512e2c3-08fa-4cec-a878-5615c666f08f None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:17.893062 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512e2c3-08fa-4cec-a878-5615c666f08f None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:17.894624 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512e2c3-08fa-4cec-a878-5615c666f08f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:17.896720 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e512e2c3-08fa-4cec-a878-5615c666f08f None admin] RBAC: Adding query filter params (role.id=156d234cfe9d4844933886f1ba568694, user.id=66e7ee5ae73e4723b79b9810ef646d0d, scope.system=all) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:17.897296 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e512e2c3-08fa-4cec-a878-5615c666f08f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:17.899381 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e512e2c3-08fa-4cec-a878-5615c666f08f None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:17.907001 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 159/309] 10.222.0.36 () {56 vars in 1308 bytes} [Thu May 19 16:28:17 2022] GET /identity/v3/role_assignments?user.id=66e7ee5ae73e4723b79b9810ef646d0d&scope.system=all&role.id=156d234cfe9d4844933886f1ba568694 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:17.987964 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af449be1-7594-486e-9767-6b0682aebecd None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:18.022588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609259a-df92-4282-896e-be3af53e8ec1 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:18.023187 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609259a-df92-4282-896e-be3af53e8ec1 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:18.023540 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609259a-df92-4282-896e-be3af53e8ec1 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:18.025237 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 160/310] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 16:28:18.032726 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a19642-856e-44a5-88e4-bf021ac29625 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:18.033108 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a19642-856e-44a5-88e4-bf021ac29625 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:18.033344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a19642-856e-44a5-88e4-bf021ac29625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:18.036772 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae1476-c7e4-4f7c-a2b3-c7c9f19fddb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:18.037798 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae1476-c7e4-4f7c-a2b3-c7c9f19fddb2 None admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:18.038080 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae1476-c7e4-4f7c-a2b3-c7c9f19fddb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:18.038359 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae1476-c7e4-4f7c-a2b3-c7c9f19fddb2 None admin] PATH_INFO: `/v3/system/users/46a4ef31fc9a49b7a971148550fab7dd/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:18.040170 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfae1476-c7e4-4f7c-a2b3-c7c9f19fddb2 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=46a4ef31fc9a49b7a971148550fab7dd, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:18.042573 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfae1476-c7e4-4f7c-a2b3-c7c9f19fddb2 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:18.055648 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-80a19642-856e-44a5-88e4-bf021ac29625 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:18.055847 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 151/311] 10.222.0.36 () {58 vars in 1334 bytes} [Thu May 19 16:28:17 2022] PUT /identity/v3/system/users/46a4ef31fc9a49b7a971148550fab7dd/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:28:18.057547 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-80a19642-856e-44a5-88e4-bf021ac29625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:18.108843 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:18.154224 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 161/312] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 16:28:18.177821 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae1476-c7e4-4f7c-a2b3-c7c9f19fddb2 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:18.257044 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:18.299788 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae1476-c7e4-4f7c-a2b3-c7c9f19fddb2 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:18.336745 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a769584-16fd-4837-8295-ae6e645c7540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:18.337674 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a769584-16fd-4837-8295-ae6e645c7540 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:18.337968 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a769584-16fd-4837-8295-ae6e645c7540 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:18.338251 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a769584-16fd-4837-8295-ae6e645c7540 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:18.350485 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a769584-16fd-4837-8295-ae6e645c7540 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:28:18.358363 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-6a769584-16fd-4837-8295-ae6e645c7540 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 16:28:18.358363 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:18.358363 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:18.358363 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:18.358363 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:18.358363 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:18.358363 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:18.358363 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:18.358363 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:18.358363 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:18.358363 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:18.358363 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:18.358363 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:28:18.358363 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:28:18.358363 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:28:18.358363 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:28:18.359255 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:28:18.359255 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:28:18.359255 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:28:18.359255 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:28:18.359255 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:18.359255 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:18.359255 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:18.359255 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:18.359255 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:18.359255 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:18.359255 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:18.359255 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:18.359255 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:18.359255 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:18.359255 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:18.359255 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:18.360065 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:18.360065 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:18.360065 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:18.360065 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:18.360065 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:28:18.360065 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:28:18.360065 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:28:18.360065 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:28:18.360065 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:28:18.360065 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:28:18.360065 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 16:28:18.360065 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:18.360065 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 152/313] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:28:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:18.366143 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a19642-856e-44a5-88e4-bf021ac29625 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:18.469273 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a19642-856e-44a5-88e4-bf021ac29625 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:18.504478 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c4c788-7cf2-412e-843d-15c72d83343a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:18.505177 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4c788-7cf2-412e-843d-15c72d83343a None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:18.505469 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4c788-7cf2-412e-843d-15c72d83343a None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:18.505747 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4c788-7cf2-412e-843d-15c72d83343a None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:18.506512 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c4c788-7cf2-412e-843d-15c72d83343a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:18.506822 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c4c788-7cf2-412e-843d-15c72d83343a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:18.508294 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c4c788-7cf2-412e-843d-15c72d83343a None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:18.517284 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 162/314] 10.222.0.36 () {56 vars in 1101 bytes} [Thu May 19 16:28:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:18.522977 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a769584-16fd-4837-8295-ae6e645c7540 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:18.642604 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a769584-16fd-4837-8295-ae6e645c7540 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:18.672435 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:18.682321 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa97efe-05e0-41d3-aa33-de7e39ab8142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:18.683613 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa97efe-05e0-41d3-aa33-de7e39ab8142 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:18.684134 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa97efe-05e0-41d3-aa33-de7e39ab8142 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:18.684628 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa97efe-05e0-41d3-aa33-de7e39ab8142 None admin] PATH_INFO: `/v3/users/24b89cd5cc53406db6fdbcbe2b60ad7b` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:18.686130 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa97efe-05e0-41d3-aa33-de7e39ab8142 None admin] RBAC: Authorizing `identity:get_user(user_id=24b89cd5cc53406db6fdbcbe2b60ad7b)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:18.687801 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa97efe-05e0-41d3-aa33-de7e39ab8142 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:18.690055 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 153/315] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:28:18 2022] GET /identity/v3/users/24b89cd5cc53406db6fdbcbe2b60ad7b => generated 297 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:18.698270 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c4c788-7cf2-412e-843d-15c72d83343a None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:18.729693 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:18.803078 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c4c788-7cf2-412e-843d-15c72d83343a None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:18.836730 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8a312b-5640-4d7a-a359-ada77ba098b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:18.837438 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a312b-5640-4d7a-a359-ada77ba098b1 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:18.837725 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a312b-5640-4d7a-a359-ada77ba098b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:18.838003 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a312b-5640-4d7a-a359-ada77ba098b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:18.908283 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8a312b-5640-4d7a-a359-ada77ba098b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:18.910404 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8a312b-5640-4d7a-a359-ada77ba098b1 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:18.948426 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:19.026898 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 163/316] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:28:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 16:28:19.034314 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa97efe-05e0-41d3-aa33-de7e39ab8142 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:19.149482 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa97efe-05e0-41d3-aa33-de7e39ab8142 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:19.188663 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee960fc-bf87-4795-a8b7-866d321ec72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:19.189574 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee960fc-bf87-4795-a8b7-866d321ec72d None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:19.190400 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee960fc-bf87-4795-a8b7-866d321ec72d None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:19.190685 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee960fc-bf87-4795-a8b7-866d321ec72d None admin] PATH_INFO: `/v3/projects/230decc93e3a483aa3d5cc33c0f662a9` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:19.191914 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee960fc-bf87-4795-a8b7-866d321ec72d None admin] RBAC: Authorizing `identity:get_project(project_id=230decc93e3a483aa3d5cc33c0f662a9)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:19.193407 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee960fc-bf87-4795-a8b7-866d321ec72d None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:19.195435 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 154/317] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:28:19 2022] GET /identity/v3/projects/230decc93e3a483aa3d5cc33c0f662a9 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:19.201322 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8a312b-5640-4d7a-a359-ada77ba098b1 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:19.316384 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8a312b-5640-4d7a-a359-ada77ba098b1 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:19.348676 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:19.355646 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff8f682-28df-4b60-9069-d5b98c40faaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:19.356886 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff8f682-28df-4b60-9069-d5b98c40faaf None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:19.357194 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff8f682-28df-4b60-9069-d5b98c40faaf None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:19.357479 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff8f682-28df-4b60-9069-d5b98c40faaf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:19.358403 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff8f682-28df-4b60-9069-d5b98c40faaf None admin] RBAC: Adding query filter params (role.id=156d234cfe9d4844933886f1ba568694, scope.project.id=230decc93e3a483aa3d5cc33c0f662a9, user.id=24b89cd5cc53406db6fdbcbe2b60ad7b) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:19.358725 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff8f682-28df-4b60-9069-d5b98c40faaf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:19.360244 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff8f682-28df-4b60-9069-d5b98c40faaf None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:19.369383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 164/318] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:28:19 2022] GET /identity/v3/role_assignments?user.id=24b89cd5cc53406db6fdbcbe2b60ad7b&scope.project.id=230decc93e3a483aa3d5cc33c0f662a9&role.id=156d234cfe9d4844933886f1ba568694 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:19.768313 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba6f24a-6a1c-40f5-b7bd-9ff7109e0106 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:19.768620 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba6f24a-6a1c-40f5-b7bd-9ff7109e0106 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:19.768894 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba6f24a-6a1c-40f5-b7bd-9ff7109e0106 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:19.770481 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 155/319] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:19.776783 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70546ad8-88bf-4004-bc1c-6aa81f0c7f9a None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:19.777077 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70546ad8-88bf-4004-bc1c-6aa81f0c7f9a None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:19.777413 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70546ad8-88bf-4004-bc1c-6aa81f0c7f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:19.794705 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-70546ad8-88bf-4004-bc1c-6aa81f0c7f9a None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:19.795933 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-70546ad8-88bf-4004-bc1c-6aa81f0c7f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:19.866516 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 165/320] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:28:19.889649 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba6f24a-6a1c-40f5-b7bd-9ff7109e0106 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:19.959249 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:20.001219 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba6f24a-6a1c-40f5-b7bd-9ff7109e0106 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:20.040015 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e8bde-b33f-4a7a-90b2-e8a289ad6402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:20.040761 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e8bde-b33f-4a7a-90b2-e8a289ad6402 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:20.041054 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e8bde-b33f-4a7a-90b2-e8a289ad6402 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:20.041370 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e8bde-b33f-4a7a-90b2-e8a289ad6402 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:20.042677 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026e8bde-b33f-4a7a-90b2-e8a289ad6402 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:20.044177 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026e8bde-b33f-4a7a-90b2-e8a289ad6402 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:20.046122 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 156/321] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:28:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:20.052299 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70546ad8-88bf-4004-bc1c-6aa81f0c7f9a None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:20.183418 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70546ad8-88bf-4004-bc1c-6aa81f0c7f9a None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:20.209771 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:20.217304 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d175495-bb02-4ad0-aee8-435590ef2fc3 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:20.217795 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d175495-bb02-4ad0-aee8-435590ef2fc3 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:20.217895 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d175495-bb02-4ad0-aee8-435590ef2fc3 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:20.219431 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 157/322] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 16:28:20.220534 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed976bd-fa3d-4b06-9aa4-68907e882aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:20.221804 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed976bd-fa3d-4b06-9aa4-68907e882aa4 None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:20.222087 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed976bd-fa3d-4b06-9aa4-68907e882aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:20.222361 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed976bd-fa3d-4b06-9aa4-68907e882aa4 None admin] PATH_INFO: `/v3/groups` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:20.223051 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed976bd-fa3d-4b06-9aa4-68907e882aa4 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:20.224581 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed976bd-fa3d-4b06-9aa4-68907e882aa4 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:20.224846 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f3362-a3e4-4f28-a9bf-56c68b00b3df None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:20.225140 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f3362-a3e4-4f28-a9bf-56c68b00b3df None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:20.225455 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f3362-a3e4-4f28-a9bf-56c68b00b3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:20.241606 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 166/323] 10.222.0.36 () {60 vars in 1134 bytes} [Thu May 19 16:28:20 2022] POST /identity/v3/groups => generated 226 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:28:20.251754 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-572f3362-a3e4-4f28-a9bf-56c68b00b3df None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:20.253008 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-572f3362-a3e4-4f28-a9bf-56c68b00b3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:20.333590 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 158/324] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 16:28:20.356677 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed976bd-fa3d-4b06-9aa4-68907e882aa4 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:20.479765 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed976bd-fa3d-4b06-9aa4-68907e882aa4 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:20.500873 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136eb0fc-ef45-4e91-ab4c-03cf4f516161 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:20.501246 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136eb0fc-ef45-4e91-ab4c-03cf4f516161 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:20.501491 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136eb0fc-ef45-4e91-ab4c-03cf4f516161 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:20.503552 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 159/325] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:20.508942 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e7997-e296-4f32-8c01-387f58940693 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:20.509270 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e7997-e296-4f32-8c01-387f58940693 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:20.509553 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e7997-e296-4f32-8c01-387f58940693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:20.517332 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aab80d-9975-4afe-b033-433c6ea0931d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:20.518712 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aab80d-9975-4afe-b033-433c6ea0931d None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:20.519000 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aab80d-9975-4afe-b033-433c6ea0931d None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:20.519272 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aab80d-9975-4afe-b033-433c6ea0931d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:20.529442 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-b32e7997-e296-4f32-8c01-387f58940693 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:20.530640 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-b32e7997-e296-4f32-8c01-387f58940693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:20.538616 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aab80d-9975-4afe-b033-433c6ea0931d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:28:20.547147 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-45aab80d-9975-4afe-b033-433c6ea0931d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 16:28:20.547147 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:20.547147 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:20.547147 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:20.547147 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:20.547147 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:20.547147 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:20.547147 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:20.547147 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:20.547147 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:20.547147 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:20.547147 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:20.547147 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:28:20.547147 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:28:20.547147 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:28:20.547147 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:28:20.548290 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:28:20.548290 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:28:20.548290 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:28:20.548290 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:28:20.548290 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:20.548290 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:20.548290 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:20.548290 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:20.548290 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:20.548290 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:20.548290 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:20.548290 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:28:20.548290 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:20.548290 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:20.548290 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:20.548290 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:20.549363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:20.549363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:20.549363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:20.549363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:20.549363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:28:20.549363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:28:20.549363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:28:20.549363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:28:20.549363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:28:20.549363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:28:20.549363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 16:28:20.549363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:28:20.551606 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 167/326] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:28:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:20.557629 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aab80d-9975-4afe-b033-433c6ea0931d None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:20.607998 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:20.621842 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:20.625954 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 160/327] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 16:28:20.650323 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32e7997-e296-4f32-8c01-387f58940693 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:20.691855 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aab80d-9975-4afe-b033-433c6ea0931d None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:20.729120 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72b2034-871a-42db-a3eb-82d2f2fd534c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:20.729822 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b2034-871a-42db-a3eb-82d2f2fd534c None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:20.730112 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b2034-871a-42db-a3eb-82d2f2fd534c None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:20.730387 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b2034-871a-42db-a3eb-82d2f2fd534c None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:20.731213 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72b2034-871a-42db-a3eb-82d2f2fd534c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:20.731525 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72b2034-871a-42db-a3eb-82d2f2fd534c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:20.733002 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72b2034-871a-42db-a3eb-82d2f2fd534c None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:20.743189 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 168/328] 10.222.0.36 () {56 vars in 1101 bytes} [Thu May 19 16:28:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:28:20.751410 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72b2034-871a-42db-a3eb-82d2f2fd534c None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:20.825614 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32e7997-e296-4f32-8c01-387f58940693 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:20.867510 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72b2034-871a-42db-a3eb-82d2f2fd534c None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:20.878584 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:20.892625 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb046c-8f88-4636-9635-aadfb0524211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:20.894412 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb046c-8f88-4636-9635-aadfb0524211 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:20.895183 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb046c-8f88-4636-9635-aadfb0524211 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:20.896264 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb046c-8f88-4636-9635-aadfb0524211 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:20.908621 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd36eaf-f8eb-4fb2-a7da-6b671b866c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:20.909543 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd36eaf-f8eb-4fb2-a7da-6b671b866c98 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:20.909829 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd36eaf-f8eb-4fb2-a7da-6b671b866c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:20.910104 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd36eaf-f8eb-4fb2-a7da-6b671b866c98 None admin] PATH_INFO: `/v3/users/66e7ee5ae73e4723b79b9810ef646d0d` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:20.911477 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd36eaf-f8eb-4fb2-a7da-6b671b866c98 None admin] RBAC: Authorizing `identity:get_user(user_id=66e7ee5ae73e4723b79b9810ef646d0d)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:20.913334 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd36eaf-f8eb-4fb2-a7da-6b671b866c98 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:20.915144 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 169/329] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:28:20 2022] GET /identity/v3/users/66e7ee5ae73e4723b79b9810ef646d0d => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:28:20.919281 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfb046c-8f88-4636-9635-aadfb0524211 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:28:20.929168 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd36eaf-f8eb-4fb2-a7da-6b671b866c98 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:20.935553 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-bbfb046c-8f88-4636-9635-aadfb0524211 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 16:28:20.935553 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:20.935553 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:20.935553 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:20.935553 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:20.935553 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:20.935553 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:20.935553 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:20.935553 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:20.935553 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:20.935553 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:20.935553 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:20.935553 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:28:20.935553 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:28:20.935553 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:28:20.935553 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:28:20.937283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:28:20.937283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:28:20.937283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:28:20.937283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:28:20.937283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:20.937283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:20.937283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:20.937283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:20.937283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:20.937283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:20.937283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:20.937283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:20.937283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:20.937283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:20.937283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:20.937283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:20.938664 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:20.938664 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:20.938664 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:20.938664 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:20.938664 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:28:20.938664 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:28:20.938664 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:28:20.938664 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:28:20.938664 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:28:20.938664 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:28:20.938664 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 16:28:20.938664 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:20.943875 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 161/330] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:28:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 16:28:20.949865 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb046c-8f88-4636-9635-aadfb0524211 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:21.045510 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd36eaf-f8eb-4fb2-a7da-6b671b866c98 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:21.057620 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:21.096154 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8bbbfb-b8c1-45cd-ac93-cabba03dcea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:21.097053 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8bbbfb-b8c1-45cd-ac93-cabba03dcea1 None admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:21.097378 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8bbbfb-b8c1-45cd-ac93-cabba03dcea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:21.097665 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8bbbfb-b8c1-45cd-ac93-cabba03dcea1 None admin] PATH_INFO: `/v3/system/users/66e7ee5ae73e4723b79b9810ef646d0d/roles/156d234cfe9d4844933886f1ba568694` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:21.099224 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8bbbfb-b8c1-45cd-ac93-cabba03dcea1 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=66e7ee5ae73e4723b79b9810ef646d0d, role_id=156d234cfe9d4844933886f1ba568694)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:21.100640 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8bbbfb-b8c1-45cd-ac93-cabba03dcea1 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:21.114336 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb046c-8f88-4636-9635-aadfb0524211 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:21.114566 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 170/331] 10.222.0.36 () {58 vars in 1334 bytes} [Thu May 19 16:28:20 2022] PUT /identity/v3/system/users/66e7ee5ae73e4723b79b9810ef646d0d/roles/156d234cfe9d4844933886f1ba568694 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:28:21.168039 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b6285-2a70-4811-8a4f-565945b0d353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:21.168870 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b6285-2a70-4811-8a4f-565945b0d353 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:21.169243 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b6285-2a70-4811-8a4f-565945b0d353 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:21.169532 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b6285-2a70-4811-8a4f-565945b0d353 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:21.170373 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b6285-2a70-4811-8a4f-565945b0d353 None admin] RBAC: Adding query filter params (name=member) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:21.170728 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b6285-2a70-4811-8a4f-565945b0d353 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:21.172222 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b6285-2a70-4811-8a4f-565945b0d353 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:21.183755 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 162/332] 10.222.0.36 () {56 vars in 1101 bytes} [Thu May 19 16:28:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:21.189057 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8bbbfb-b8c1-45cd-ac93-cabba03dcea1 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:21.242912 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:21.315572 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8bbbfb-b8c1-45cd-ac93-cabba03dcea1 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:21.358345 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c2b04-fba5-47c6-b9ca-6106e0b3d1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:21.359254 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c2b04-fba5-47c6-b9ca-6106e0b3d1d5 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:21.359538 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c2b04-fba5-47c6-b9ca-6106e0b3d1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:21.359815 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c2b04-fba5-47c6-b9ca-6106e0b3d1d5 None admin] PATH_INFO: `/v3/users/46a4ef31fc9a49b7a971148550fab7dd` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:21.361088 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c2b04-fba5-47c6-b9ca-6106e0b3d1d5 None admin] RBAC: Authorizing `identity:get_user(user_id=46a4ef31fc9a49b7a971148550fab7dd)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:21.363205 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c2b04-fba5-47c6-b9ca-6106e0b3d1d5 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:21.365409 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 171/333] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:28:21 2022] GET /identity/v3/users/46a4ef31fc9a49b7a971148550fab7dd => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:21.371318 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b6285-2a70-4811-8a4f-565945b0d353 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:21.494799 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b6285-2a70-4811-8a4f-565945b0d353 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:21.509799 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:21.532985 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4fe550-7058-4595-b2ce-4be1b794535b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:21.534232 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4fe550-7058-4595-b2ce-4be1b794535b None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:21.534521 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4fe550-7058-4595-b2ce-4be1b794535b None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:21.534798 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4fe550-7058-4595-b2ce-4be1b794535b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:21.535649 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4fe550-7058-4595-b2ce-4be1b794535b None admin] RBAC: Adding query filter params (role.id=0e98fd94a6d24e46b9fe09a22a7df04a, user.id=46a4ef31fc9a49b7a971148550fab7dd, scope.system=all) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:21.535970 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4fe550-7058-4595-b2ce-4be1b794535b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:21.537600 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4fe550-7058-4595-b2ce-4be1b794535b None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:21.556117 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 163/334] 10.222.0.36 () {56 vars in 1308 bytes} [Thu May 19 16:28:21 2022] GET /identity/v3/role_assignments?user.id=46a4ef31fc9a49b7a971148550fab7dd&scope.system=all&role.id=0e98fd94a6d24e46b9fe09a22a7df04a => generated 510 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:23.651246 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca81b50e-921e-40a3-9e7e-e5ee0e91f7c5 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:23.652267 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca81b50e-921e-40a3-9e7e-e5ee0e91f7c5 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:23.652766 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca81b50e-921e-40a3-9e7e-e5ee0e91f7c5 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:23.654415 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 172/335] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:23.660610 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d8666-bf9a-4683-be8f-345cc9ac354d None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:23.661587 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d8666-bf9a-4683-be8f-345cc9ac354d None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:23.662076 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d8666-bf9a-4683-be8f-345cc9ac354d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:23.680546 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-9b0d8666-bf9a-4683-be8f-345cc9ac354d None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:23.682171 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-9b0d8666-bf9a-4683-be8f-345cc9ac354d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:23.776890 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 164/336] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:28:23.788601 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75feb2-f0bb-48d0-9d61-5d4e10041d50 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:23.789338 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75feb2-f0bb-48d0-9d61-5d4e10041d50 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:23.789859 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75feb2-f0bb-48d0-9d61-5d4e10041d50 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:23.791571 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 173/337] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:23.797271 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d02e4d-b98c-4b65-a258-4ae71ff6cd1b None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:23.797856 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d02e4d-b98c-4b65-a258-4ae71ff6cd1b None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:23.798336 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d02e4d-b98c-4b65-a258-4ae71ff6cd1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:23.822161 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-a3d02e4d-b98c-4b65-a258-4ae71ff6cd1b None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:23.823695 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-a3d02e4d-b98c-4b65-a258-4ae71ff6cd1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:23.847135 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e75feb2-f0bb-48d0-9d61-5d4e10041d50 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:23.951199 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 165/338] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:28:23.952629 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dcc554-31e8-49df-9419-0a7c5397777d None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:23.952927 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dcc554-31e8-49df-9419-0a7c5397777d None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:23.954502 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dcc554-31e8-49df-9419-0a7c5397777d None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:23.958330 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 166/339] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:23.963451 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c00c0e-be77-414f-af02-36fda68f3c0a None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:23.963934 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c00c0e-be77-414f-af02-36fda68f3c0a None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:23.964355 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c00c0e-be77-414f-af02-36fda68f3c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:23.990692 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-a2c00c0e-be77-414f-af02-36fda68f3c0a None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:23.996330 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-a2c00c0e-be77-414f-af02-36fda68f3c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:24.032362 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e75feb2-f0bb-48d0-9d61-5d4e10041d50 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:24.047063 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:24.047925 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:24.078649 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab18860-c876-42c6-b2cb-fc3cf7b9cd6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:24.080749 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab18860-c876-42c6-b2cb-fc3cf7b9cd6d None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:24.081506 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab18860-c876-42c6-b2cb-fc3cf7b9cd6d None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:24.081943 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab18860-c876-42c6-b2cb-fc3cf7b9cd6d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:24.095785 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 167/340] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 16:28:24.097830 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c24cc-39d5-4f53-9898-83d0f98e9340 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:24.098138 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c24cc-39d5-4f53-9898-83d0f98e9340 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:24.098425 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c24cc-39d5-4f53-9898-83d0f98e9340 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:24.099822 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab18860-c876-42c6-b2cb-fc3cf7b9cd6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:28:24.100110 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 168/341] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:24.100853 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3c24cc-39d5-4f53-9898-83d0f98e9340 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:24.108418 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-cab18860-c876-42c6-b2cb-fc3cf7b9cd6d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 16:28:24.108418 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:24.108418 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:24.108418 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:24.108418 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:24.108418 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:24.108418 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:24.108418 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:24.108418 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:24.108418 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:24.108418 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:24.108418 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:24.108418 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:28:24.108418 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:28:24.108418 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:28:24.108418 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:28:24.109558 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:28:24.109558 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:28:24.109558 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:28:24.109558 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:28:24.109558 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:24.109558 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:24.109558 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:24.109558 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:24.109558 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:24.109558 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:24.109558 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:24.109558 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:28:24.109558 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:24.109558 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:24.109558 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:24.109558 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:24.110567 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:24.110567 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:24.110567 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:24.110567 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:24.110567 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:28:24.110567 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:28:24.110567 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:28:24.110567 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:28:24.110567 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:28:24.110567 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:28:24.110567 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 16:28:24.110567 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:28:24.114090 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 174/342] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:28:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:24.115599 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2642020-144f-442b-af4d-36baa97c61cc None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:24.116059 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2642020-144f-442b-af4d-36baa97c61cc None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:24.116490 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2642020-144f-442b-af4d-36baa97c61cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:24.135675 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-f2642020-144f-442b-af4d-36baa97c61cc None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:24.137260 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-f2642020-144f-442b-af4d-36baa97c61cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:24.222861 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 175/343] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:28:24.223933 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2642020-144f-442b-af4d-36baa97c61cc None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:24.253968 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3c24cc-39d5-4f53-9898-83d0f98e9340 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:24.299979 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199843b0-b114-4d13-af68-214550b19ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:24.300802 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199843b0-b114-4d13-af68-214550b19ebe None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:24.301087 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199843b0-b114-4d13-af68-214550b19ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:24.301386 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199843b0-b114-4d13-af68-214550b19ebe None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:24.315571 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199843b0-b114-4d13-af68-214550b19ebe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:28:24.324582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-199843b0-b114-4d13-af68-214550b19ebe None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 16:28:24.324582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:24.324582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:24.324582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:24.324582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:24.324582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:24.324582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:24.324582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:24.324582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:24.324582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:24.324582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:24.324582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:24.324582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:28:24.324582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:28:24.324582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:28:24.324582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:28:24.326036 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:28:24.326036 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:28:24.326036 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:28:24.326036 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:28:24.326036 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:24.326036 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:24.326036 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:24.326036 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:24.326036 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:24.326036 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:24.326036 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:24.326036 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:24.326036 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:24.326036 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:24.326036 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:24.326036 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:24.326839 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:24.326839 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:24.326839 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:24.326839 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:24.326839 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:28:24.326839 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:28:24.326839 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:28:24.326839 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:28:24.326839 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:28:24.326839 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:28:24.326839 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 16:28:24.326839 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:24.328908 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 169/344] 10.222.0.36 () {56 vars in 1114 bytes} [Thu May 19 16:28:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:24.329015 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199843b0-b114-4d13-af68-214550b19ebe None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:24.353783 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:24.359353 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2642020-144f-442b-af4d-36baa97c61cc None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:24.364587 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:24.410369 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2202c2b1-e88c-4edc-b304-86251123bae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:24.411085 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202c2b1-e88c-4edc-b304-86251123bae6 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:24.411360 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202c2b1-e88c-4edc-b304-86251123bae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:24.411645 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202c2b1-e88c-4edc-b304-86251123bae6 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:24.412538 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2202c2b1-e88c-4edc-b304-86251123bae6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:24.412842 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2202c2b1-e88c-4edc-b304-86251123bae6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:24.414899 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2202c2b1-e88c-4edc-b304-86251123bae6 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:24.424405 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 176/345] 10.222.0.36 () {56 vars in 1101 bytes} [Thu May 19 16:28:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:24.425353 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2202c2b1-e88c-4edc-b304-86251123bae6 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:24.467804 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199843b0-b114-4d13-af68-214550b19ebe None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:24.510605 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c336cd78-4961-40d1-ac59-9d1383ddc2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:24.511336 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c336cd78-4961-40d1-ac59-9d1383ddc2c7 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:24.511625 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c336cd78-4961-40d1-ac59-9d1383ddc2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:24.511907 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c336cd78-4961-40d1-ac59-9d1383ddc2c7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:24.525443 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c336cd78-4961-40d1-ac59-9d1383ddc2c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:28:24.534095 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-c336cd78-4961-40d1-ac59-9d1383ddc2c7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 16:28:24.534095 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:24.534095 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:24.534095 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:24.534095 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:24.534095 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:24.534095 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:24.534095 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:24.534095 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:24.534095 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:24.534095 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:24.534095 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:24.534095 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:28:24.534095 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:28:24.534095 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:28:24.534095 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:28:24.535081 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:28:24.535081 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:28:24.535081 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:28:24.535081 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:28:24.535081 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:24.535081 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:24.535081 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:24.535081 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:24.535081 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:24.535081 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:24.535081 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:24.535081 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:24.535081 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:24.535081 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:24.535081 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:24.535081 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:24.535884 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:24.535884 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:24.535884 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:24.535884 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:24.535884 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:28:24.535884 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:28:24.535884 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:28:24.535884 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:28:24.535884 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:28:24.535884 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:28:24.535884 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 16:28:24.535884 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:24.535884 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 170/346] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:28:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:24.537227 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c336cd78-4961-40d1-ac59-9d1383ddc2c7 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:24.567068 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2202c2b1-e88c-4edc-b304-86251123bae6 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:24.612748 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d304847f-d7c7-4916-9dcf-a73c53542d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:24.614138 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304847f-d7c7-4916-9dcf-a73c53542d2d None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:24.614738 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304847f-d7c7-4916-9dcf-a73c53542d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:24.615213 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304847f-d7c7-4916-9dcf-a73c53542d2d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:24.629049 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d304847f-d7c7-4916-9dcf-a73c53542d2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:28:24.637646 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-d304847f-d7c7-4916-9dcf-a73c53542d2d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 16:28:24.637646 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:24.637646 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:24.637646 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:24.637646 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:24.637646 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:24.637646 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:24.637646 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:24.637646 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:24.637646 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:24.637646 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:24.637646 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:24.637646 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:28:24.637646 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:28:24.637646 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:28:24.637646 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:28:24.638582 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:28:24.638582 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:28:24.638582 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:28:24.638582 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:28:24.638582 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:24.638582 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:24.638582 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:24.638582 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:24.638582 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:24.638582 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:24.638582 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:24.638582 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:28:24.638582 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:24.638582 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:24.638582 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:24.638582 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:24.639368 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:24.639368 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:24.639368 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:24.639368 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:24.639368 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:28:24.639368 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:28:24.639368 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:28:24.639368 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:28:24.639368 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:28:24.639368 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:28:24.639368 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 16:28:24.639368 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:28:24.640785 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:24.643959 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 177/347] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:28:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:24.650160 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d304847f-d7c7-4916-9dcf-a73c53542d2d None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:24.662766 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c336cd78-4961-40d1-ac59-9d1383ddc2c7 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:24.678780 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:24.704763 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397094be-58ec-4134-b8d3-23e54ab7107f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:24.707500 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397094be-58ec-4134-b8d3-23e54ab7107f None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:24.707500 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397094be-58ec-4134-b8d3-23e54ab7107f None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:24.707500 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397094be-58ec-4134-b8d3-23e54ab7107f None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:24.707500 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397094be-58ec-4134-b8d3-23e54ab7107f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:24.707802 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397094be-58ec-4134-b8d3-23e54ab7107f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:24.709060 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397094be-58ec-4134-b8d3-23e54ab7107f None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:24.719665 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 171/348] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:28:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:24.720692 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397094be-58ec-4134-b8d3-23e54ab7107f None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:24.775306 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d304847f-d7c7-4916-9dcf-a73c53542d2d None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:24.812917 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c4d11-bd50-451f-962d-2a3856953e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:24.814050 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c4d11-bd50-451f-962d-2a3856953e15 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:24.815293 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c4d11-bd50-451f-962d-2a3856953e15 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:24.815774 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c4d11-bd50-451f-962d-2a3856953e15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:24.830351 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397094be-58ec-4134-b8d3-23e54ab7107f None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:24.872282 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b965785b-4a18-4b3e-9c19-18cc30ae4c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:24.872594 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b965785b-4a18-4b3e-9c19-18cc30ae4c98 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:24.872990 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b965785b-4a18-4b3e-9c19-18cc30ae4c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:24.873145 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b965785b-4a18-4b3e-9c19-18cc30ae4c98 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:24.874018 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b965785b-4a18-4b3e-9c19-18cc30ae4c98 None admin] RBAC: Adding query filter params (name=member) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:24.874339 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b965785b-4a18-4b3e-9c19-18cc30ae4c98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:24.875751 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b965785b-4a18-4b3e-9c19-18cc30ae4c98 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:24.885758 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 172/349] 10.222.0.36 () {56 vars in 1101 bytes} [Thu May 19 16:28:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:24.886693 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b965785b-4a18-4b3e-9c19-18cc30ae4c98 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:24.897566 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:24.912253 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c4d11-bd50-451f-962d-2a3856953e15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:24.915211 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c4d11-bd50-451f-962d-2a3856953e15 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:24.940500 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:25.002143 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b965785b-4a18-4b3e-9c19-18cc30ae4c98 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:25.031033 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 178/350] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:28:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 16:28:25.032700 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c4d11-bd50-451f-962d-2a3856953e15 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:25.042337 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97847154-64e3-4aec-9231-699965182f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:25.043279 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97847154-64e3-4aec-9231-699965182f0c None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:25.043717 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97847154-64e3-4aec-9231-699965182f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:25.044223 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97847154-64e3-4aec-9231-699965182f0c None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:25.045232 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97847154-64e3-4aec-9231-699965182f0c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:25.045782 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97847154-64e3-4aec-9231-699965182f0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:25.047381 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97847154-64e3-4aec-9231-699965182f0c None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:25.058913 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 173/351] 10.222.0.36 () {56 vars in 1101 bytes} [Thu May 19 16:28:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:25.060206 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97847154-64e3-4aec-9231-699965182f0c None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:25.154345 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:25.175245 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c4d11-bd50-451f-962d-2a3856953e15 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:25.205543 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:25.211212 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97847154-64e3-4aec-9231-699965182f0c None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:25.220490 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6646513-d4a3-4ac0-a5fd-56f4c7c1e1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:25.222121 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6646513-d4a3-4ac0-a5fd-56f4c7c1e1a4 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:25.222623 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6646513-d4a3-4ac0-a5fd-56f4c7c1e1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:25.223135 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6646513-d4a3-4ac0-a5fd-56f4c7c1e1a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:25.266297 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9f952-3931-418e-882e-c50a8568748f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:25.267592 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9f952-3931-418e-882e-c50a8568748f None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:25.268224 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9f952-3931-418e-882e-c50a8568748f None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:25.268825 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9f952-3931-418e-882e-c50a8568748f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:25.304470 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6646513-d4a3-4ac0-a5fd-56f4c7c1e1a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:25.306340 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6646513-d4a3-4ac0-a5fd-56f4c7c1e1a4 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:25.353057 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac9f952-3931-418e-882e-c50a8568748f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:25.354896 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac9f952-3931-418e-882e-c50a8568748f None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:25.434758 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 179/352] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:28:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 16:28:25.437596 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6646513-d4a3-4ac0-a5fd-56f4c7c1e1a4 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:25.442835 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:25.469496 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:25.494027 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 174/353] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:28:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 16:28:25.495667 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9f952-3931-418e-882e-c50a8568748f None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:25.585207 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6646513-d4a3-4ac0-a5fd-56f4c7c1e1a4 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:25.628337 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9f952-3931-418e-882e-c50a8568748f None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:25.632433 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64c5507-bb30-4124-a754-ec9ebc097f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:25.632649 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c5507-bb30-4124-a754-ec9ebc097f86 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:25.633065 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c5507-bb30-4124-a754-ec9ebc097f86 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:25.633442 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c5507-bb30-4124-a754-ec9ebc097f86 None admin] PATH_INFO: `/v3/projects/d4a456cd7f83414a895ab7982fec2a68` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:25.634749 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64c5507-bb30-4124-a754-ec9ebc097f86 None admin] RBAC: Authorizing `identity:get_project(project_id=d4a456cd7f83414a895ab7982fec2a68)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:25.636229 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64c5507-bb30-4124-a754-ec9ebc097f86 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:25.638006 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 180/354] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:28:25 2022] GET /identity/v3/projects/d4a456cd7f83414a895ab7982fec2a68 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:25.639242 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697c187-6eb7-422c-85a3-578cfb91aeda None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:25.639532 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697c187-6eb7-422c-85a3-578cfb91aeda None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:25.644883 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697c187-6eb7-422c-85a3-578cfb91aeda None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:25.646687 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 181/355] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:25.647483 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4697c187-6eb7-422c-85a3-578cfb91aeda None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:25.671819 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe99a0ee-7798-4d1f-aa45-7a46933b2862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:25.673193 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99a0ee-7798-4d1f-aa45-7a46933b2862 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:25.673494 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99a0ee-7798-4d1f-aa45-7a46933b2862 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:25.673807 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99a0ee-7798-4d1f-aa45-7a46933b2862 None admin] PATH_INFO: `/v3/users/66e7ee5ae73e4723b79b9810ef646d0d` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:25.675067 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe99a0ee-7798-4d1f-aa45-7a46933b2862 None admin] RBAC: Authorizing `identity:get_user(user_id=66e7ee5ae73e4723b79b9810ef646d0d)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:25.677183 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe99a0ee-7798-4d1f-aa45-7a46933b2862 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:25.679662 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 175/356] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:28:25 2022] GET /identity/v3/users/66e7ee5ae73e4723b79b9810ef646d0d => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:25.680926 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe99a0ee-7798-4d1f-aa45-7a46933b2862 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:25.719408 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:25.755540 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:25.772489 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4697c187-6eb7-422c-85a3-578cfb91aeda None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:25.800981 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe99a0ee-7798-4d1f-aa45-7a46933b2862 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:25.817876 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf36034e-da33-4e43-9c01-f7041cbe1de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:25.818761 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36034e-da33-4e43-9c01-f7041cbe1de5 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:25.819049 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36034e-da33-4e43-9c01-f7041cbe1de5 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:25.820018 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36034e-da33-4e43-9c01-f7041cbe1de5 None admin] PATH_INFO: `/v3/projects/d4a456cd7f83414a895ab7982fec2a68` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:25.820529 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf36034e-da33-4e43-9c01-f7041cbe1de5 None admin] RBAC: Authorizing `identity:get_project(project_id=d4a456cd7f83414a895ab7982fec2a68)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:25.822025 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf36034e-da33-4e43-9c01-f7041cbe1de5 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:25.823926 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 182/357] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:28:25 2022] GET /identity/v3/projects/d4a456cd7f83414a895ab7982fec2a68 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:25.824560 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf36034e-da33-4e43-9c01-f7041cbe1de5 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:25.841747 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ca26fe-0541-4af1-a0a5-9ce323c5d69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:25.842540 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca26fe-0541-4af1-a0a5-9ce323c5d69b None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:25.842823 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca26fe-0541-4af1-a0a5-9ce323c5d69b None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:25.843098 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca26fe-0541-4af1-a0a5-9ce323c5d69b None admin] PATH_INFO: `/v3/projects/230decc93e3a483aa3d5cc33c0f662a9` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:25.844295 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ca26fe-0541-4af1-a0a5-9ce323c5d69b None admin] RBAC: Authorizing `identity:get_project(project_id=230decc93e3a483aa3d5cc33c0f662a9)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:25.845776 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ca26fe-0541-4af1-a0a5-9ce323c5d69b None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:25.847496 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 176/358] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:28:25 2022] GET /identity/v3/projects/230decc93e3a483aa3d5cc33c0f662a9 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:25.848663 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a2c3b-b138-4f0c-a7d8-ae43470d99cd None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:25.848949 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a2c3b-b138-4f0c-a7d8-ae43470d99cd None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:25.849252 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a2c3b-b138-4f0c-a7d8-ae43470d99cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:25.868577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-ab9a2c3b-b138-4f0c-a7d8-ae43470d99cd None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:25.869800 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ab9a2c3b-b138-4f0c-a7d8-ae43470d99cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:25.953804 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 177/359] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:28:25.954598 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf36034e-da33-4e43-9c01-f7041cbe1de5 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:25.955731 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4556c0-5880-40f3-865a-4778677b7631 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:25.956032 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4556c0-5880-40f3-865a-4778677b7631 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:25.956315 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4556c0-5880-40f3-865a-4778677b7631 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:25.957943 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 178/360] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:25.958821 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4556c0-5880-40f3-865a-4778677b7631 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:25.995709 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1e6d32-23e0-41eb-a7f9-77ceda067ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:25.996683 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1e6d32-23e0-41eb-a7f9-77ceda067ae2 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:25.996988 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1e6d32-23e0-41eb-a7f9-77ceda067ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:25.997297 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1e6d32-23e0-41eb-a7f9-77ceda067ae2 None admin] PATH_INFO: `/v3/groups/9eaa3a30967242c095e3c139fb633c93` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:26.002686 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:26.004257 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1e6d32-23e0-41eb-a7f9-77ceda067ae2 None admin] RBAC: Authorizing `identity:get_group(group_id=9eaa3a30967242c095e3c139fb633c93)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:26.005769 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1e6d32-23e0-41eb-a7f9-77ceda067ae2 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:26.007480 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 183/361] 10.222.0.36 () {56 vars in 1180 bytes} [Thu May 19 16:28:25 2022] GET /identity/v3/groups/9eaa3a30967242c095e3c139fb633c93 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:26.008261 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1e6d32-23e0-41eb-a7f9-77ceda067ae2 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:26.031998 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:26.094053 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4556c0-5880-40f3-865a-4778677b7631 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:26.134727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1e6d32-23e0-41eb-a7f9-77ceda067ae2 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:26.135659 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a1cef8-34ab-4e6d-9450-92b50e4b0fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:26.136328 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a1cef8-34ab-4e6d-9450-92b50e4b0fcf None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:26.136611 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a1cef8-34ab-4e6d-9450-92b50e4b0fcf None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:26.136891 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a1cef8-34ab-4e6d-9450-92b50e4b0fcf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:26.137827 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a1cef8-34ab-4e6d-9450-92b50e4b0fcf None admin] RBAC: Adding query filter params (role.id=156d234cfe9d4844933886f1ba568694, user.id=66e7ee5ae73e4723b79b9810ef646d0d, scope.system=all) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:26.138137 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a1cef8-34ab-4e6d-9450-92b50e4b0fcf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:26.139580 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a1cef8-34ab-4e6d-9450-92b50e4b0fcf None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:26.155454 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 179/362] 10.222.0.36 () {56 vars in 1308 bytes} [Thu May 19 16:28:25 2022] GET /identity/v3/role_assignments?user.id=66e7ee5ae73e4723b79b9810ef646d0d&scope.system=all&role.id=156d234cfe9d4844933886f1ba568694 => generated 510 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:26.156360 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a1cef8-34ab-4e6d-9450-92b50e4b0fcf None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:26.183702 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26454c32-6923-4b3b-a55d-1acd66ed5738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:26.186170 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26454c32-6923-4b3b-a55d-1acd66ed5738 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:26.186783 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26454c32-6923-4b3b-a55d-1acd66ed5738 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:26.187300 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26454c32-6923-4b3b-a55d-1acd66ed5738 None admin] PATH_INFO: `/v3/groups/9eaa3a30967242c095e3c139fb633c93` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:26.189003 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26454c32-6923-4b3b-a55d-1acd66ed5738 None admin] RBAC: Authorizing `identity:get_group(group_id=9eaa3a30967242c095e3c139fb633c93)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:26.191468 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26454c32-6923-4b3b-a55d-1acd66ed5738 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:26.198476 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 184/363] 10.222.0.36 () {56 vars in 1180 bytes} [Thu May 19 16:28:26 2022] GET /identity/v3/groups/9eaa3a30967242c095e3c139fb633c93 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:26.198476 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9f743-4d5a-4409-b910-b7a0a85fdbe2 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:26.198476 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9f743-4d5a-4409-b910-b7a0a85fdbe2 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:26.198476 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9f743-4d5a-4409-b910-b7a0a85fdbe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:26.222071 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-14b9f743-4d5a-4409-b910-b7a0a85fdbe2 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:26.223338 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-14b9f743-4d5a-4409-b910-b7a0a85fdbe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:26.286178 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:26.302046 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a1cef8-34ab-4e6d-9450-92b50e4b0fcf None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:26.315723 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 185/364] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:28:26.316596 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b9f743-4d5a-4409-b910-b7a0a85fdbe2 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:26.338031 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:26.351384 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b14748-b189-4bc0-b99c-3f06fb2eb8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:26.352276 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b14748-b189-4bc0-b99c-3f06fb2eb8b0 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:26.352659 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b14748-b189-4bc0-b99c-3f06fb2eb8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:26.353033 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b14748-b189-4bc0-b99c-3f06fb2eb8b0 None admin] PATH_INFO: `/v3/groups/9eaa3a30967242c095e3c139fb633c93` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:26.354717 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b14748-b189-4bc0-b99c-3f06fb2eb8b0 None admin] RBAC: Authorizing `identity:get_group(group_id=9eaa3a30967242c095e3c139fb633c93)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:26.356182 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b14748-b189-4bc0-b99c-3f06fb2eb8b0 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:26.362425 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 180/365] 10.222.0.36 () {56 vars in 1180 bytes} [Thu May 19 16:28:26 2022] GET /identity/v3/groups/9eaa3a30967242c095e3c139fb633c93 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:26.362425 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b14748-b189-4bc0-b99c-3f06fb2eb8b0 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:26.448174 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b9f743-4d5a-4409-b910-b7a0a85fdbe2 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:26.473293 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b14748-b189-4bc0-b99c-3f06fb2eb8b0 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:26.490716 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d602b7ed-5545-45bd-91e8-204b0394cd7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:26.491538 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602b7ed-5545-45bd-91e8-204b0394cd7d None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:26.491819 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602b7ed-5545-45bd-91e8-204b0394cd7d None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:26.492099 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602b7ed-5545-45bd-91e8-204b0394cd7d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:26.504179 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d602b7ed-5545-45bd-91e8-204b0394cd7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:28:26.512347 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f0e80e-dfdc-49df-91d1-8823cbbcd5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:26.512524 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-d602b7ed-5545-45bd-91e8-204b0394cd7d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 16:28:26.512524 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:26.512524 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:26.512524 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:26.512524 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:26.512524 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:26.512524 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:26.512524 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:26.512524 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:26.512524 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:26.512524 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:26.512524 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:26.512524 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:28:26.512524 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:28:26.512524 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:28:26.512524 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:28:26.513417 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:28:26.513417 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:28:26.513417 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:28:26.513417 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:28:26.513417 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:26.513417 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:26.513417 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:26.513417 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:26.513417 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:26.513417 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:26.513417 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:26.513417 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:28:26.513417 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:26.513417 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:26.513417 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:26.513417 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:26.514206 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:26.514206 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:26.514206 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:26.514206 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:26.514206 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:28:26.514206 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:28:26.514206 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:28:26.514206 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:28:26.514206 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:28:26.514206 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:28:26.514206 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 16:28:26.514206 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:28:26.514809 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0e80e-dfdc-49df-91d1-8823cbbcd5f0 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:26.514809 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0e80e-dfdc-49df-91d1-8823cbbcd5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:26.514809 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0e80e-dfdc-49df-91d1-8823cbbcd5f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:26.514991 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 186/366] 10.222.0.36 () {56 vars in 1114 bytes} [Thu May 19 16:28:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:26.514991 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d602b7ed-5545-45bd-91e8-204b0394cd7d None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:26.517776 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f0e80e-dfdc-49df-91d1-8823cbbcd5f0 None admin] RBAC: Adding query filter params (group.id=9eaa3a30967242c095e3c139fb633c93, role.id=0e98fd94a6d24e46b9fe09a22a7df04a, scope.project.id=d4a456cd7f83414a895ab7982fec2a68) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:26.518165 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f0e80e-dfdc-49df-91d1-8823cbbcd5f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:26.519688 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f0e80e-dfdc-49df-91d1-8823cbbcd5f0 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:26.526131 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 181/367] 10.222.0.36 () {56 vars in 1376 bytes} [Thu May 19 16:28:26 2022] GET /identity/v3/role_assignments?group.id=9eaa3a30967242c095e3c139fb633c93&scope.project.id=d4a456cd7f83414a895ab7982fec2a68&role.id=0e98fd94a6d24e46b9fe09a22a7df04a => generated 260 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:26.526944 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f0e80e-dfdc-49df-91d1-8823cbbcd5f0 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:26.560523 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:26.631970 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:26.658855 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d602b7ed-5545-45bd-91e8-204b0394cd7d None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:26.683423 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f0e80e-dfdc-49df-91d1-8823cbbcd5f0 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:26.713093 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018e10b7-c548-4053-9fdd-70b927a2d1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:26.713743 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e10b7-c548-4053-9fdd-70b927a2d1ee None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:26.714035 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e10b7-c548-4053-9fdd-70b927a2d1ee None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:26.715473 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e10b7-c548-4053-9fdd-70b927a2d1ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:26.716211 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018e10b7-c548-4053-9fdd-70b927a2d1ee None admin] RBAC: Adding query filter params (group.id=9eaa3a30967242c095e3c139fb633c93, role.id=50f62a33eb434682baa7529619fe14b0, scope.project.id=d4a456cd7f83414a895ab7982fec2a68) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:26.716527 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018e10b7-c548-4053-9fdd-70b927a2d1ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:26.718054 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018e10b7-c548-4053-9fdd-70b927a2d1ee None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:26.724495 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 187/368] 10.222.0.36 () {56 vars in 1376 bytes} [Thu May 19 16:28:26 2022] GET /identity/v3/role_assignments?group.id=9eaa3a30967242c095e3c139fb633c93&scope.project.id=d4a456cd7f83414a895ab7982fec2a68&role.id=50f62a33eb434682baa7529619fe14b0 => generated 260 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:26.728577 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018e10b7-c548-4053-9fdd-70b927a2d1ee None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:26.739698 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0ca27e-bb5d-4c7d-80db-f04f76778bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:26.740530 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ca27e-bb5d-4c7d-80db-f04f76778bab None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:26.740817 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ca27e-bb5d-4c7d-80db-f04f76778bab None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:26.741093 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ca27e-bb5d-4c7d-80db-f04f76778bab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:26.758499 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0ca27e-bb5d-4c7d-80db-f04f76778bab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:28:26.767199 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-bf0ca27e-bb5d-4c7d-80db-f04f76778bab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:28:26.767199 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:26.767199 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:26.767199 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:26.767199 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:26.767199 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:26.767199 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:26.767199 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:26.767199 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:26.767199 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:26.767199 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:26.767199 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:26.767199 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:28:26.767199 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:28:26.767199 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:28:26.767199 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:28:26.768729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:28:26.768729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:28:26.768729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:28:26.768729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:28:26.768729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:26.768729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:26.768729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:26.768729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:26.768729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:26.768729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:26.768729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:26.768729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:26.768729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:26.768729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:26.768729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:26.768729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:26.769945 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:26.769945 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:26.769945 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:26.769945 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:26.769945 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:28:26.769945 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:28:26.769945 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:28:26.769945 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:28:26.769945 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:28:26.769945 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:28:26.769945 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:28:26.769945 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:26.769945 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 182/369] 10.222.0.36 () {56 vars in 1096 bytes} [Thu May 19 16:28:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:26.770606 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0ca27e-bb5d-4c7d-80db-f04f76778bab None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:26.880430 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018e10b7-c548-4053-9fdd-70b927a2d1ee None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:26.886041 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:26.909831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0ca27e-bb5d-4c7d-80db-f04f76778bab None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:26.933346 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81bf099-3941-40f0-808d-05af8daed5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:26.933677 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81bf099-3941-40f0-808d-05af8daed5b2 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:26.933907 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81bf099-3941-40f0-808d-05af8daed5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:26.934188 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81bf099-3941-40f0-808d-05af8daed5b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:26.935048 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81bf099-3941-40f0-808d-05af8daed5b2 None admin] RBAC: Adding query filter params (group.id=9eaa3a30967242c095e3c139fb633c93, role.id=0e98fd94a6d24e46b9fe09a22a7df04a, scope.project.id=230decc93e3a483aa3d5cc33c0f662a9) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:26.935365 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81bf099-3941-40f0-808d-05af8daed5b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:26.936849 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81bf099-3941-40f0-808d-05af8daed5b2 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:26.944282 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 188/370] 10.222.0.36 () {56 vars in 1376 bytes} [Thu May 19 16:28:26 2022] GET /identity/v3/role_assignments?group.id=9eaa3a30967242c095e3c139fb633c93&scope.project.id=230decc93e3a483aa3d5cc33c0f662a9&role.id=0e98fd94a6d24e46b9fe09a22a7df04a => generated 260 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:26.945174 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81bf099-3941-40f0-808d-05af8daed5b2 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:26.957467 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:26.965113 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8ffc4-babb-411e-8f26-ca7720a5cd47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:26.966415 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8ffc4-babb-411e-8f26-ca7720a5cd47 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:26.966982 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8ffc4-babb-411e-8f26-ca7720a5cd47 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:26.967466 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8ffc4-babb-411e-8f26-ca7720a5cd47 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:26.968473 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8ffc4-babb-411e-8f26-ca7720a5cd47 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:26.969010 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8ffc4-babb-411e-8f26-ca7720a5cd47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:26.971738 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8ffc4-babb-411e-8f26-ca7720a5cd47 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:26.981912 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 183/371] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:28:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:26.990294 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8ffc4-babb-411e-8f26-ca7720a5cd47 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:27.091873 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81bf099-3941-40f0-808d-05af8daed5b2 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:27.150813 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a764c-e0e5-4667-957d-c632348d8460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:27.151430 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a764c-e0e5-4667-957d-c632348d8460 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:27.151723 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a764c-e0e5-4667-957d-c632348d8460 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:27.152004 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a764c-e0e5-4667-957d-c632348d8460 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:27.152780 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6a764c-e0e5-4667-957d-c632348d8460 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:27.153094 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6a764c-e0e5-4667-957d-c632348d8460 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:27.154552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6a764c-e0e5-4667-957d-c632348d8460 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:27.171030 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8ffc4-babb-411e-8f26-ca7720a5cd47 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:27.178581 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 189/372] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:28:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:27.185188 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a764c-e0e5-4667-957d-c632348d8460 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:27.248757 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbeafa2-d020-4ed8-b42f-b24f7ea000ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:27.249476 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbeafa2-d020-4ed8-b42f-b24f7ea000ec None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:27.249764 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbeafa2-d020-4ed8-b42f-b24f7ea000ec None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:27.250045 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbeafa2-d020-4ed8-b42f-b24f7ea000ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:27.278857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:27.288867 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:27.328374 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a764c-e0e5-4667-957d-c632348d8460 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:27.366598 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9370fb27-2c89-4e96-bce4-0e93ab3606b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:27.367272 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370fb27-2c89-4e96-bce4-0e93ab3606b3 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:27.367666 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370fb27-2c89-4e96-bce4-0e93ab3606b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:27.367952 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370fb27-2c89-4e96-bce4-0e93ab3606b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:27.370261 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbeafa2-d020-4ed8-b42f-b24f7ea000ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:27.371755 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbeafa2-d020-4ed8-b42f-b24f7ea000ec None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:27.444332 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9370fb27-2c89-4e96-bce4-0e93ab3606b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:27.445794 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9370fb27-2c89-4e96-bce4-0e93ab3606b3 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:27.520565 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:27.537783 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 184/373] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:28:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 16:28:27.555560 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbeafa2-d020-4ed8-b42f-b24f7ea000ec None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:27.569266 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 190/374] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:28:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 16:28:27.571575 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9370fb27-2c89-4e96-bce4-0e93ab3606b3 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:27.628878 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:27.684356 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9370fb27-2c89-4e96-bce4-0e93ab3606b3 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:27.722236 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f347910-87d9-4a9a-bf65-5c6fe084ca15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:27.723040 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f347910-87d9-4a9a-bf65-5c6fe084ca15 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:27.723328 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f347910-87d9-4a9a-bf65-5c6fe084ca15 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:27.723603 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f347910-87d9-4a9a-bf65-5c6fe084ca15 None admin] PATH_INFO: `/v3/projects/8d5de9b2c50341a8b78fb369c7765787` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:27.725085 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f347910-87d9-4a9a-bf65-5c6fe084ca15 None admin] RBAC: Authorizing `identity:get_project(project_id=8d5de9b2c50341a8b78fb369c7765787)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:27.726587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f347910-87d9-4a9a-bf65-5c6fe084ca15 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:27.728445 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 191/375] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:28:27 2022] GET /identity/v3/projects/8d5de9b2c50341a8b78fb369c7765787 => generated 344 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:28:27.733946 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f347910-87d9-4a9a-bf65-5c6fe084ca15 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:27.737180 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbeafa2-d020-4ed8-b42f-b24f7ea000ec None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:27.777702 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:27.799147 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ed8f7-fa27-4cb0-b359-14e33d6bcc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:27.800018 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ed8f7-fa27-4cb0-b359-14e33d6bcc2a None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:27.800301 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ed8f7-fa27-4cb0-b359-14e33d6bcc2a None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:27.800576 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ed8f7-fa27-4cb0-b359-14e33d6bcc2a None admin] PATH_INFO: `/v3/projects/230decc93e3a483aa3d5cc33c0f662a9` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:27.805998 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226ed8f7-fa27-4cb0-b359-14e33d6bcc2a None admin] RBAC: Authorizing `identity:get_project(project_id=230decc93e3a483aa3d5cc33c0f662a9)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:27.807547 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226ed8f7-fa27-4cb0-b359-14e33d6bcc2a None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:27.809747 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 185/376] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:28:27 2022] GET /identity/v3/projects/230decc93e3a483aa3d5cc33c0f662a9 => generated 302 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:28:27.825516 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ed8f7-fa27-4cb0-b359-14e33d6bcc2a None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:27.857620 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f347910-87d9-4a9a-bf65-5c6fe084ca15 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:27.901499 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8df956-3707-4959-9a68-429c93a4d323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:27.902492 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8df956-3707-4959-9a68-429c93a4d323 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:27.902785 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8df956-3707-4959-9a68-429c93a4d323 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:27.903064 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8df956-3707-4959-9a68-429c93a4d323 None admin] PATH_INFO: `/v3/groups/46c5c11a3ae94d158114a2ea156c9b6f` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:27.910327 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8df956-3707-4959-9a68-429c93a4d323 None admin] RBAC: Authorizing `identity:get_group(group_id=46c5c11a3ae94d158114a2ea156c9b6f)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:27.911969 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8df956-3707-4959-9a68-429c93a4d323 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:27.918594 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 192/377] 10.222.0.36 () {56 vars in 1180 bytes} [Thu May 19 16:28:27 2022] GET /identity/v3/groups/46c5c11a3ae94d158114a2ea156c9b6f => generated 229 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:28:27.920054 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8df956-3707-4959-9a68-429c93a4d323 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:28.008608 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ed8f7-fa27-4cb0-b359-14e33d6bcc2a None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:28.045534 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:28.071692 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c86fc-1743-4a0f-b5f4-14ae307bda96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:28.072434 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c86fc-1743-4a0f-b5f4-14ae307bda96 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:28.072725 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c86fc-1743-4a0f-b5f4-14ae307bda96 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:28.072997 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c86fc-1743-4a0f-b5f4-14ae307bda96 None admin] PATH_INFO: `/v3/groups/9eaa3a30967242c095e3c139fb633c93` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:28.074406 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20c86fc-1743-4a0f-b5f4-14ae307bda96 None admin] RBAC: Authorizing `identity:get_group(group_id=9eaa3a30967242c095e3c139fb633c93)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:28.075879 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20c86fc-1743-4a0f-b5f4-14ae307bda96 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:28.084484 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 186/378] 10.222.0.36 () {56 vars in 1180 bytes} [Thu May 19 16:28:27 2022] GET /identity/v3/groups/9eaa3a30967242c095e3c139fb633c93 => generated 226 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:28:28.084484 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c86fc-1743-4a0f-b5f4-14ae307bda96 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:28.107501 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:28.111298 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8df956-3707-4959-9a68-429c93a4d323 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:28.182855 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d4fb6c-8f0e-4a04-a6d5-26d35835a8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:28.183916 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d4fb6c-8f0e-4a04-a6d5-26d35835a8b4 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:28.184417 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d4fb6c-8f0e-4a04-a6d5-26d35835a8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:28.184897 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d4fb6c-8f0e-4a04-a6d5-26d35835a8b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:28.185969 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d4fb6c-8f0e-4a04-a6d5-26d35835a8b4 None admin] RBAC: Adding query filter params (group.id=46c5c11a3ae94d158114a2ea156c9b6f, role.id=e2915fe109244a07a9be96bd2b219fdd, scope.project.id=8d5de9b2c50341a8b78fb369c7765787) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:28.186485 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d4fb6c-8f0e-4a04-a6d5-26d35835a8b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:28.188141 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d4fb6c-8f0e-4a04-a6d5-26d35835a8b4 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:28.196882 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c86fc-1743-4a0f-b5f4-14ae307bda96 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:28.201800 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 193/379] 10.222.0.36 () {56 vars in 1376 bytes} [Thu May 19 16:28:27 2022] GET /identity/v3/role_assignments?group.id=46c5c11a3ae94d158114a2ea156c9b6f&scope.project.id=8d5de9b2c50341a8b78fb369c7765787&role.id=e2915fe109244a07a9be96bd2b219fdd => generated 260 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:28.266021 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a584bc-36d1-4276-8b00-2ab4596cf4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:28.266618 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a584bc-36d1-4276-8b00-2ab4596cf4ee None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:28.266904 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a584bc-36d1-4276-8b00-2ab4596cf4ee None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:28.267182 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a584bc-36d1-4276-8b00-2ab4596cf4ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:28.268073 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a584bc-36d1-4276-8b00-2ab4596cf4ee None admin] RBAC: Adding query filter params (group.id=9eaa3a30967242c095e3c139fb633c93, role.id=50f62a33eb434682baa7529619fe14b0, scope.project.id=230decc93e3a483aa3d5cc33c0f662a9) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:28.268389 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a584bc-36d1-4276-8b00-2ab4596cf4ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:28.269876 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a584bc-36d1-4276-8b00-2ab4596cf4ee None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:28.285691 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 187/380] 10.222.0.36 () {56 vars in 1376 bytes} [Thu May 19 16:28:28 2022] GET /identity/v3/role_assignments?group.id=9eaa3a30967242c095e3c139fb633c93&scope.project.id=230decc93e3a483aa3d5cc33c0f662a9&role.id=50f62a33eb434682baa7529619fe14b0 => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:29.208002 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438e7c48-c6d1-4e4d-b01b-e404360dcb4f None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:29.208491 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438e7c48-c6d1-4e4d-b01b-e404360dcb4f None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:29.208491 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438e7c48-c6d1-4e4d-b01b-e404360dcb4f None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:29.210170 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 194/381] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:29.215762 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee32ea-f9d2-4967-9501-1ee3c177d441 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:29.216198 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee32ea-f9d2-4967-9501-1ee3c177d441 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:29.216379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee32ea-f9d2-4967-9501-1ee3c177d441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:29.234032 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-bfee32ea-f9d2-4967-9501-1ee3c177d441 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:29.235245 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-bfee32ea-f9d2-4967-9501-1ee3c177d441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:29.288672 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:29.322602 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 188/382] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 16:28:29.364408 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438e7c48-c6d1-4e4d-b01b-e404360dcb4f None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:29.506038 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438e7c48-c6d1-4e4d-b01b-e404360dcb4f None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:29.553866 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aa57c9-d9e7-471d-ad42-e4e5cfa0fca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:29.554682 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa57c9-d9e7-471d-ad42-e4e5cfa0fca4 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:29.554971 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa57c9-d9e7-471d-ad42-e4e5cfa0fca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:29.555253 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa57c9-d9e7-471d-ad42-e4e5cfa0fca4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:29.569992 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aa57c9-d9e7-471d-ad42-e4e5cfa0fca4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:28:29.577762 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-02aa57c9-d9e7-471d-ad42-e4e5cfa0fca4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 16:28:29.577762 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:29.577762 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:29.577762 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:29.577762 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:29.577762 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:29.577762 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:29.577762 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:29.577762 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:29.577762 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:29.577762 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:29.577762 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:29.577762 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:28:29.577762 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:28:29.577762 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:28:29.577762 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:28:29.578624 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:28:29.578624 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:28:29.578624 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:28:29.578624 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:28:29.578624 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:29.578624 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:29.578624 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:29.578624 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:29.578624 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:29.578624 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:29.578624 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:29.578624 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:28:29.578624 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:29.578624 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:29.578624 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:29.578624 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:29.579401 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:29.579401 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:29.579401 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:29.579401 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:29.579401 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:28:29.579401 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:28:29.579401 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:28:29.579401 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:28:29.579401 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:28:29.579401 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:28:29.579401 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 16:28:29.579401 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:28:29.581024 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 195/383] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:28:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:29.585801 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee32ea-f9d2-4967-9501-1ee3c177d441 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:29.697293 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2eb3b8-bda1-459e-8d71-871e4a9dda46 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:29.697493 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2eb3b8-bda1-459e-8d71-871e4a9dda46 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:29.697783 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2eb3b8-bda1-459e-8d71-871e4a9dda46 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:29.699188 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 196/384] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 16:28:29.711099 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b2fec6-acac-47a1-a0fc-21108b5668c6 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:29.711396 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b2fec6-acac-47a1-a0fc-21108b5668c6 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:29.711664 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b2fec6-acac-47a1-a0fc-21108b5668c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:29.727686 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee32ea-f9d2-4967-9501-1ee3c177d441 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:29.734670 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-a0b2fec6-acac-47a1-a0fc-21108b5668c6 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:29.736271 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-a0b2fec6-acac-47a1-a0fc-21108b5668c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:29.758060 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:29.799049 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d2ed3c-a6bb-48d5-8fc0-f67a369d26f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:29.799757 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d2ed3c-a6bb-48d5-8fc0-f67a369d26f6 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:29.800044 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d2ed3c-a6bb-48d5-8fc0-f67a369d26f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:29.800321 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d2ed3c-a6bb-48d5-8fc0-f67a369d26f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:29.801121 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d2ed3c-a6bb-48d5-8fc0-f67a369d26f6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:29.801462 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d2ed3c-a6bb-48d5-8fc0-f67a369d26f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:29.802915 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d2ed3c-a6bb-48d5-8fc0-f67a369d26f6 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:29.821692 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 197/385] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 16:28:29.824753 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 189/386] 10.222.0.36 () {56 vars in 1101 bytes} [Thu May 19 16:28:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:29.833409 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b2fec6-acac-47a1-a0fc-21108b5668c6 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:29.844131 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d2ed3c-a6bb-48d5-8fc0-f67a369d26f6 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:29.953327 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b2fec6-acac-47a1-a0fc-21108b5668c6 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:29.973636 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:30.024355 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71650c1-38aa-4641-830f-a017e759e07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:30.025169 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71650c1-38aa-4641-830f-a017e759e07d None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:30.025479 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71650c1-38aa-4641-830f-a017e759e07d None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:30.025755 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71650c1-38aa-4641-830f-a017e759e07d None admin] PATH_INFO: `/v3/groups/9eaa3a30967242c095e3c139fb633c93` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:30.027121 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71650c1-38aa-4641-830f-a017e759e07d None admin] RBAC: Authorizing `identity:get_group(group_id=9eaa3a30967242c095e3c139fb633c93)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:30.028763 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71650c1-38aa-4641-830f-a017e759e07d None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:30.030795 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 198/387] 10.222.0.36 () {56 vars in 1180 bytes} [Thu May 19 16:28:29 2022] GET /identity/v3/groups/9eaa3a30967242c095e3c139fb633c93 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:30.032319 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19938c7-7fba-48e7-86a9-16140858de8c None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:30.032606 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19938c7-7fba-48e7-86a9-16140858de8c None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:30.032885 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19938c7-7fba-48e7-86a9-16140858de8c None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:30.038342 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 199/388] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:30.039323 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19938c7-7fba-48e7-86a9-16140858de8c None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:30.069978 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:30.070388 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d2ed3c-a6bb-48d5-8fc0-f67a369d26f6 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:30.118229 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8c17c-29cd-4396-9c6d-af736502f9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:30.119543 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8c17c-29cd-4396-9c6d-af736502f9d2 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:30.120593 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8c17c-29cd-4396-9c6d-af736502f9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:30.121223 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8c17c-29cd-4396-9c6d-af736502f9d2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:30.133135 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe8c17c-29cd-4396-9c6d-af736502f9d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:28:30.141821 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-4fe8c17c-29cd-4396-9c6d-af736502f9d2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 16:28:30.141821 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:30.141821 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:30.141821 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:30.141821 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:30.141821 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:30.141821 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:30.141821 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:30.141821 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:30.141821 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:30.141821 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:30.141821 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:30.141821 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:28:30.141821 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:28:30.141821 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:28:30.141821 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:28:30.142864 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:28:30.142864 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:28:30.142864 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:28:30.142864 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:28:30.142864 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:30.142864 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:30.142864 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:30.142864 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:30.142864 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:30.142864 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:30.142864 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:30.142864 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:30.142864 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:30.142864 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:30.142864 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:30.142864 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:30.143655 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:30.143655 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:30.143655 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:30.143655 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:30.143655 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:28:30.143655 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:28:30.143655 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:28:30.143655 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:28:30.143655 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:28:30.143655 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:28:30.143655 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 16:28:30.143655 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:30.146376 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 190/389] 10.222.0.36 () {56 vars in 1114 bytes} [Thu May 19 16:28:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 16:28:30.152418 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18678c1-6493-4d9a-8855-2e11ee0160aa None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:30.152931 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18678c1-6493-4d9a-8855-2e11ee0160aa None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:30.153525 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18678c1-6493-4d9a-8855-2e11ee0160aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:30.168513 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19938c7-7fba-48e7-86a9-16140858de8c None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:30.171522 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-b18678c1-6493-4d9a-8855-2e11ee0160aa None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:30.172960 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-b18678c1-6493-4d9a-8855-2e11ee0160aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:30.215891 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d08be-518d-4162-a9d1-6dc4dd43a0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:30.216876 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d08be-518d-4162-a9d1-6dc4dd43a0f4 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:30.217250 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d08be-518d-4162-a9d1-6dc4dd43a0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:30.217594 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d08be-518d-4162-a9d1-6dc4dd43a0f4 None admin] PATH_INFO: `/v3/projects/d4a456cd7f83414a895ab7982fec2a68` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:30.218868 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d08be-518d-4162-a9d1-6dc4dd43a0f4 None admin] RBAC: Authorizing `identity:get_project(project_id=d4a456cd7f83414a895ab7982fec2a68)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:30.221481 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d08be-518d-4162-a9d1-6dc4dd43a0f4 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:30.223963 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 200/390] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:28:30 2022] GET /identity/v3/projects/d4a456cd7f83414a895ab7982fec2a68 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:30.224896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d08be-518d-4162-a9d1-6dc4dd43a0f4 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:30.254159 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 191/391] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:28:30.262418 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18678c1-6493-4d9a-8855-2e11ee0160aa None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:30.309233 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:30.338355 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:30.351685 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d08be-518d-4162-a9d1-6dc4dd43a0f4 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:30.393463 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18678c1-6493-4d9a-8855-2e11ee0160aa None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:30.400112 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6d7147-938d-48bb-a30d-83f908855419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:30.401040 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d7147-938d-48bb-a30d-83f908855419 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:30.401584 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d7147-938d-48bb-a30d-83f908855419 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:30.402037 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d7147-938d-48bb-a30d-83f908855419 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:30.403177 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6d7147-938d-48bb-a30d-83f908855419 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:30.403678 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6d7147-938d-48bb-a30d-83f908855419 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:30.405411 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6d7147-938d-48bb-a30d-83f908855419 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:30.416093 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 201/392] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:28:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:30.417268 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6d7147-938d-48bb-a30d-83f908855419 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:30.441073 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471c3a39-6457-4fa6-b792-4ef76a5f4f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:30.442385 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c3a39-6457-4fa6-b792-4ef76a5f4f46 None admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:30.443401 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c3a39-6457-4fa6-b792-4ef76a5f4f46 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:30.443865 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c3a39-6457-4fa6-b792-4ef76a5f4f46 None admin] PATH_INFO: `/v3/projects/d4a456cd7f83414a895ab7982fec2a68/groups/9eaa3a30967242c095e3c139fb633c93/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:30.447238 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471c3a39-6457-4fa6-b792-4ef76a5f4f46 None admin] RBAC: Authorizing `identity:create_grant(project_id=d4a456cd7f83414a895ab7982fec2a68, group_id=9eaa3a30967242c095e3c139fb633c93, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:30.449492 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471c3a39-6457-4fa6-b792-4ef76a5f4f46 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:30.468442 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 192/393] 10.222.0.36 () {58 vars in 1442 bytes} [Thu May 19 16:28:30 2022] PUT /identity/v3/projects/d4a456cd7f83414a895ab7982fec2a68/groups/9eaa3a30967242c095e3c139fb633c93/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:28:30.469696 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471c3a39-6457-4fa6-b792-4ef76a5f4f46 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:30.584828 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:30.600560 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6d7147-938d-48bb-a30d-83f908855419 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:30.606799 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471c3a39-6457-4fa6-b792-4ef76a5f4f46 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:30.651880 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494920c-2ea2-4193-bca5-18036a2bf8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:30.653073 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494920c-2ea2-4193-bca5-18036a2bf8ee None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:30.653725 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494920c-2ea2-4193-bca5-18036a2bf8ee None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:30.654182 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494920c-2ea2-4193-bca5-18036a2bf8ee None admin] PATH_INFO: `/v3/groups/9eaa3a30967242c095e3c139fb633c93` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:30.655943 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2494920c-2ea2-4193-bca5-18036a2bf8ee None admin] RBAC: Authorizing `identity:get_group(group_id=9eaa3a30967242c095e3c139fb633c93)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:30.657852 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2494920c-2ea2-4193-bca5-18036a2bf8ee None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:30.661725 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 193/394] 10.222.0.36 () {56 vars in 1180 bytes} [Thu May 19 16:28:30 2022] GET /identity/v3/groups/9eaa3a30967242c095e3c139fb633c93 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:30.669919 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494920c-2ea2-4193-bca5-18036a2bf8ee None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:30.685679 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b2bd44-da9f-4d23-8422-23e240b31d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:30.686563 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b2bd44-da9f-4d23-8422-23e240b31d70 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:30.686853 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b2bd44-da9f-4d23-8422-23e240b31d70 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:30.687133 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b2bd44-da9f-4d23-8422-23e240b31d70 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:30.709914 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b2bd44-da9f-4d23-8422-23e240b31d70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:28:30.726688 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-27b2bd44-da9f-4d23-8422-23e240b31d70 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 16:28:30.726688 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:30.726688 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:30.726688 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:30.726688 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:30.726688 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:30.726688 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:30.726688 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:30.726688 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:30.726688 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:30.726688 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:30.726688 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:30.726688 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:28:30.726688 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:28:30.726688 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:28:30.726688 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:28:30.727712 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:28:30.727712 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:28:30.727712 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:28:30.727712 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:28:30.727712 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:30.727712 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:30.727712 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:30.727712 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:30.727712 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:30.727712 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:30.727712 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:30.727712 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:28:30.727712 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:30.727712 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:30.727712 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:30.727712 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:30.728503 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:30.728503 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:30.728503 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:30.728503 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:30.728503 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:28:30.728503 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:28:30.728503 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:28:30.728503 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:28:30.728503 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:28:30.728503 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:28:30.728503 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 16:28:30.728503 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:28:30.729727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 202/395] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:28:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:30.737599 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b2bd44-da9f-4d23-8422-23e240b31d70 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:30.763022 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:30.868138 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494920c-2ea2-4193-bca5-18036a2bf8ee None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:30.899224 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b2bd44-da9f-4d23-8422-23e240b31d70 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:30.920702 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657314f-c53b-4773-8b63-ee8c149de48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:30.921663 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657314f-c53b-4773-8b63-ee8c149de48d None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:30.921957 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657314f-c53b-4773-8b63-ee8c149de48d None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:30.922365 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657314f-c53b-4773-8b63-ee8c149de48d None admin] PATH_INFO: `/v3/projects/d4a456cd7f83414a895ab7982fec2a68` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:30.924348 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5657314f-c53b-4773-8b63-ee8c149de48d None admin] RBAC: Authorizing `identity:get_project(project_id=d4a456cd7f83414a895ab7982fec2a68)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:30.925877 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5657314f-c53b-4773-8b63-ee8c149de48d None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:30.928632 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 194/396] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:28:30 2022] GET /identity/v3/projects/d4a456cd7f83414a895ab7982fec2a68 => generated 298 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:28:30.930185 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aadd290-87a9-42df-b01c-18837f21c91f None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:30.930618 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aadd290-87a9-42df-b01c-18837f21c91f None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:30.930924 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aadd290-87a9-42df-b01c-18837f21c91f None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:30.933024 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 195/397] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:30.938337 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aadd290-87a9-42df-b01c-18837f21c91f None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:30.946003 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ae7dd-b91e-4410-84fa-40665df5d4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:30.946890 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ae7dd-b91e-4410-84fa-40665df5d4c9 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:30.947322 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ae7dd-b91e-4410-84fa-40665df5d4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:30.947773 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ae7dd-b91e-4410-84fa-40665df5d4c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:30.948675 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046ae7dd-b91e-4410-84fa-40665df5d4c9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:30.949144 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046ae7dd-b91e-4410-84fa-40665df5d4c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:30.950749 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046ae7dd-b91e-4410-84fa-40665df5d4c9 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:30.960881 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 203/398] 10.222.0.36 () {56 vars in 1101 bytes} [Thu May 19 16:28:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:30.962449 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f999f20b-d8f4-44b4-8bbc-48b6e6ecca22 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:30.962912 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f999f20b-d8f4-44b4-8bbc-48b6e6ecca22 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:30.963336 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f999f20b-d8f4-44b4-8bbc-48b6e6ecca22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:30.969079 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:30.982737 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-f999f20b-d8f4-44b4-8bbc-48b6e6ecca22 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:30.984064 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-f999f20b-d8f4-44b4-8bbc-48b6e6ecca22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:31.054721 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aadd290-87a9-42df-b01c-18837f21c91f None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:31.068283 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 204/399] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:28:31.069983 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448118a-1398-4ae2-a82c-4e7cbde6dadd None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:31.070459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448118a-1398-4ae2-a82c-4e7cbde6dadd None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:31.070972 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448118a-1398-4ae2-a82c-4e7cbde6dadd None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:31.072766 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 205/400] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:31.073702 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3448118a-1398-4ae2-a82c-4e7cbde6dadd None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:31.083294 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:31.107303 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d1eb69-184b-4cb6-92a3-a25220bfbc46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:31.108209 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d1eb69-184b-4cb6-92a3-a25220bfbc46 None admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:31.108480 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d1eb69-184b-4cb6-92a3-a25220bfbc46 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:31.108759 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d1eb69-184b-4cb6-92a3-a25220bfbc46 None admin] PATH_INFO: `/v3/projects/d4a456cd7f83414a895ab7982fec2a68/groups/9eaa3a30967242c095e3c139fb633c93/roles/50f62a33eb434682baa7529619fe14b0` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:31.111096 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d1eb69-184b-4cb6-92a3-a25220bfbc46 None admin] RBAC: Authorizing `identity:create_grant(project_id=d4a456cd7f83414a895ab7982fec2a68, group_id=9eaa3a30967242c095e3c139fb633c93, role_id=50f62a33eb434682baa7529619fe14b0)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:31.112543 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d1eb69-184b-4cb6-92a3-a25220bfbc46 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:31.128182 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 196/401] 10.222.0.36 () {58 vars in 1442 bytes} [Thu May 19 16:28:30 2022] PUT /identity/v3/projects/d4a456cd7f83414a895ab7982fec2a68/groups/9eaa3a30967242c095e3c139fb633c93/roles/50f62a33eb434682baa7529619fe14b0 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:28:31.129503 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfcfc1d-5da7-40d0-b17a-4964417871ce None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:31.129798 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfcfc1d-5da7-40d0-b17a-4964417871ce None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:31.130734 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfcfc1d-5da7-40d0-b17a-4964417871ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:31.162155 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-7cfcfc1d-5da7-40d0-b17a-4964417871ce None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:31.163442 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-7cfcfc1d-5da7-40d0-b17a-4964417871ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:31.219283 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3448118a-1398-4ae2-a82c-4e7cbde6dadd None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:31.257370 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 197/402] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:28:31.258033 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfcfc1d-5da7-40d0-b17a-4964417871ce None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:31.270591 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310540c2-7063-4105-a339-71ff734a29fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:31.273145 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310540c2-7063-4105-a339-71ff734a29fd None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:31.273456 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310540c2-7063-4105-a339-71ff734a29fd None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:31.273739 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310540c2-7063-4105-a339-71ff734a29fd None admin] PATH_INFO: `/v3/groups/9eaa3a30967242c095e3c139fb633c93` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:31.278135 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310540c2-7063-4105-a339-71ff734a29fd None admin] RBAC: Authorizing `identity:get_group(group_id=9eaa3a30967242c095e3c139fb633c93)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:31.279673 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310540c2-7063-4105-a339-71ff734a29fd None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:31.282184 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 206/403] 10.222.0.36 () {56 vars in 1180 bytes} [Thu May 19 16:28:31 2022] GET /identity/v3/groups/9eaa3a30967242c095e3c139fb633c93 => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:31.282999 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310540c2-7063-4105-a339-71ff734a29fd None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:31.295538 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:31.383152 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:31.405617 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfcfc1d-5da7-40d0-b17a-4964417871ce None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:31.431468 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310540c2-7063-4105-a339-71ff734a29fd None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:31.451218 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e67e65d-4c17-4379-b47d-ae5f474ecdb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:31.452423 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67e65d-4c17-4379-b47d-ae5f474ecdb6 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:31.452980 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67e65d-4c17-4379-b47d-ae5f474ecdb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:31.453308 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67e65d-4c17-4379-b47d-ae5f474ecdb6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:31.470245 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e67e65d-4c17-4379-b47d-ae5f474ecdb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:28:31.477280 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3296cac-9ecd-4fb9-850d-cb358585a1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:31.478214 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3296cac-9ecd-4fb9-850d-cb358585a1cb None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:31.478608 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3296cac-9ecd-4fb9-850d-cb358585a1cb None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:31.479013 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3296cac-9ecd-4fb9-850d-cb358585a1cb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:31.480126 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-0e67e65d-4c17-4379-b47d-ae5f474ecdb6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:28:31.480126 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:31.480126 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:31.480126 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:31.480126 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:31.480126 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:31.480126 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:31.480126 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:31.480126 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:31.480126 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:31.480126 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:31.480126 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:31.480126 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:28:31.480126 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:28:31.480126 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:28:31.480126 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:28:31.481263 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:28:31.481263 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:28:31.481263 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:28:31.481263 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:28:31.481263 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:31.481263 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:31.481263 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:31.481263 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:31.481263 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:31.481263 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:31.481263 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:31.481263 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:31.481263 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:31.481263 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:31.481263 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:31.481263 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:31.482062 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:31.482062 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:31.482062 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:31.482062 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:31.482062 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:28:31.482062 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:28:31.482062 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:28:31.482062 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:28:31.482062 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:28:31.482062 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:28:31.482062 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:28:31.482062 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:31.485637 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 198/404] 10.222.0.36 () {56 vars in 1096 bytes} [Thu May 19 16:28:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:31.486851 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e67e65d-4c17-4379-b47d-ae5f474ecdb6 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:31.499464 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3296cac-9ecd-4fb9-850d-cb358585a1cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:28:31.508641 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-f3296cac-9ecd-4fb9-850d-cb358585a1cb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 16:28:31.508641 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:31.508641 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:31.508641 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:31.508641 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:31.508641 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:31.508641 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:31.508641 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:31.508641 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:31.508641 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:31.508641 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:31.508641 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:31.508641 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:28:31.508641 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:28:31.508641 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:28:31.508641 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:28:31.509587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:28:31.509587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:28:31.509587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:28:31.509587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:28:31.509587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:31.509587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:31.509587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:31.509587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:31.509587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:31.509587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:31.509587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:31.509587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:28:31.509587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:31.509587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:31.509587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:31.509587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:31.510375 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:31.510375 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:31.510375 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:31.510375 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:31.510375 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:28:31.510375 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:28:31.510375 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:28:31.510375 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:28:31.510375 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:28:31.510375 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:28:31.510375 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 16:28:31.510375 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:28:31.517505 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 207/405] 10.222.0.36 () {56 vars in 1114 bytes} [Thu May 19 16:28:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:31.518674 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3296cac-9ecd-4fb9-850d-cb358585a1cb None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:31.575364 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:31.603670 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e67e65d-4c17-4379-b47d-ae5f474ecdb6 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:31.629551 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3296cac-9ecd-4fb9-850d-cb358585a1cb None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:31.641050 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a44b83-73bb-4852-91a4-67577d67a3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:31.641782 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a44b83-73bb-4852-91a4-67577d67a3de None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:31.642076 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a44b83-73bb-4852-91a4-67577d67a3de None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:31.642393 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a44b83-73bb-4852-91a4-67577d67a3de None admin] PATH_INFO: `/v3/projects/230decc93e3a483aa3d5cc33c0f662a9` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:31.643718 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a44b83-73bb-4852-91a4-67577d67a3de None admin] RBAC: Authorizing `identity:get_project(project_id=230decc93e3a483aa3d5cc33c0f662a9)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:31.646267 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a44b83-73bb-4852-91a4-67577d67a3de None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:31.652797 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 199/406] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:28:31 2022] GET /identity/v3/projects/230decc93e3a483aa3d5cc33c0f662a9 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:31.652797 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a44b83-73bb-4852-91a4-67577d67a3de None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:31.669001 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:31.675637 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8494c694-a688-4c6b-b4da-3634e83485fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:31.676634 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8494c694-a688-4c6b-b4da-3634e83485fb None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:31.677116 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8494c694-a688-4c6b-b4da-3634e83485fb None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:31.677644 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8494c694-a688-4c6b-b4da-3634e83485fb None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:31.678626 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8494c694-a688-4c6b-b4da-3634e83485fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:31.679131 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8494c694-a688-4c6b-b4da-3634e83485fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:31.680717 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8494c694-a688-4c6b-b4da-3634e83485fb None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:31.695924 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 208/407] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:28:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:31.697096 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8494c694-a688-4c6b-b4da-3634e83485fb None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:31.765524 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a44b83-73bb-4852-91a4-67577d67a3de None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:31.802301 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699b9e68-7d23-48bd-aa0d-ec3a5e13498e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:31.803378 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699b9e68-7d23-48bd-aa0d-ec3a5e13498e None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:31.803861 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699b9e68-7d23-48bd-aa0d-ec3a5e13498e None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:31.804406 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699b9e68-7d23-48bd-aa0d-ec3a5e13498e None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:31.805484 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699b9e68-7d23-48bd-aa0d-ec3a5e13498e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:31.806022 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699b9e68-7d23-48bd-aa0d-ec3a5e13498e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:31.807589 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699b9e68-7d23-48bd-aa0d-ec3a5e13498e None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:31.812124 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8494c694-a688-4c6b-b4da-3634e83485fb None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:31.821509 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 200/408] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:28:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:31.822804 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699b9e68-7d23-48bd-aa0d-ec3a5e13498e None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:31.828022 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:31.852467 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0163c220-8dea-4e4d-b166-14a90a7a42e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:31.853778 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163c220-8dea-4e4d-b166-14a90a7a42e4 None admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:31.854208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163c220-8dea-4e4d-b166-14a90a7a42e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:31.854760 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163c220-8dea-4e4d-b166-14a90a7a42e4 None admin] PATH_INFO: `/v3/projects/230decc93e3a483aa3d5cc33c0f662a9/groups/9eaa3a30967242c095e3c139fb633c93/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:31.857041 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0163c220-8dea-4e4d-b166-14a90a7a42e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=230decc93e3a483aa3d5cc33c0f662a9, group_id=9eaa3a30967242c095e3c139fb633c93, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:31.858788 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0163c220-8dea-4e4d-b166-14a90a7a42e4 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:31.877210 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 209/409] 10.222.0.36 () {58 vars in 1442 bytes} [Thu May 19 16:28:31 2022] PUT /identity/v3/projects/230decc93e3a483aa3d5cc33c0f662a9/groups/9eaa3a30967242c095e3c139fb633c93/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:28:31.878463 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0163c220-8dea-4e4d-b166-14a90a7a42e4 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:31.952786 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:31.989039 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699b9e68-7d23-48bd-aa0d-ec3a5e13498e None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:32.006030 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0163c220-8dea-4e4d-b166-14a90a7a42e4 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:32.046080 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de2fa8-5caa-49e8-85a3-67ae1a907a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:32.047237 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de2fa8-5caa-49e8-85a3-67ae1a907a24 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:32.047673 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de2fa8-5caa-49e8-85a3-67ae1a907a24 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:32.048087 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de2fa8-5caa-49e8-85a3-67ae1a907a24 None admin] PATH_INFO: `/v3/groups/9eaa3a30967242c095e3c139fb633c93` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:32.050224 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94de2fa8-5caa-49e8-85a3-67ae1a907a24 None admin] RBAC: Authorizing `identity:get_group(group_id=9eaa3a30967242c095e3c139fb633c93)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:32.052007 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94de2fa8-5caa-49e8-85a3-67ae1a907a24 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:32.054305 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 210/410] 10.222.0.36 () {56 vars in 1180 bytes} [Thu May 19 16:28:31 2022] GET /identity/v3/groups/9eaa3a30967242c095e3c139fb633c93 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:32.065847 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbc907e-a0b8-4120-bd4e-3dbd99c0f287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:32.066989 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc907e-a0b8-4120-bd4e-3dbd99c0f287 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:32.067422 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc907e-a0b8-4120-bd4e-3dbd99c0f287 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:32.067815 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc907e-a0b8-4120-bd4e-3dbd99c0f287 None admin] PATH_INFO: `/v3/groups/46c5c11a3ae94d158114a2ea156c9b6f` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:32.069982 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbc907e-a0b8-4120-bd4e-3dbd99c0f287 None admin] RBAC: Authorizing `identity:get_group(group_id=46c5c11a3ae94d158114a2ea156c9b6f)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:32.074482 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbc907e-a0b8-4120-bd4e-3dbd99c0f287 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:32.074482 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 201/411] 10.222.0.36 () {56 vars in 1180 bytes} [Thu May 19 16:28:31 2022] GET /identity/v3/groups/46c5c11a3ae94d158114a2ea156c9b6f => generated 229 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:32.082335 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de2fa8-5caa-49e8-85a3-67ae1a907a24 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:32.083509 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbc907e-a0b8-4120-bd4e-3dbd99c0f287 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:32.212874 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:32.246553 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbc907e-a0b8-4120-bd4e-3dbd99c0f287 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:32.256137 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de2fa8-5caa-49e8-85a3-67ae1a907a24 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:32.287613 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd4649-000f-4c26-89f9-2bd416ac735b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:32.288130 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:32.288248 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd4649-000f-4c26-89f9-2bd416ac735b None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:32.288248 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd4649-000f-4c26-89f9-2bd416ac735b None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:32.288248 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd4649-000f-4c26-89f9-2bd416ac735b None admin] PATH_INFO: `/v3/projects/230decc93e3a483aa3d5cc33c0f662a9` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:32.289311 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dd4649-000f-4c26-89f9-2bd416ac735b None admin] RBAC: Authorizing `identity:get_project(project_id=230decc93e3a483aa3d5cc33c0f662a9)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:32.290900 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dd4649-000f-4c26-89f9-2bd416ac735b None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:32.293012 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 202/412] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:28:32 2022] GET /identity/v3/projects/230decc93e3a483aa3d5cc33c0f662a9 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:32.299008 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd4649-000f-4c26-89f9-2bd416ac735b None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:32.302275 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c92ad-754d-4440-8999-87918e0b3f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:32.305037 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c92ad-754d-4440-8999-87918e0b3f21 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:32.305651 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c92ad-754d-4440-8999-87918e0b3f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:32.306199 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c92ad-754d-4440-8999-87918e0b3f21 None admin] PATH_INFO: `/v3/projects/8d5de9b2c50341a8b78fb369c7765787` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:32.308457 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792c92ad-754d-4440-8999-87918e0b3f21 None admin] RBAC: Authorizing `identity:get_project(project_id=8d5de9b2c50341a8b78fb369c7765787)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:32.315059 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792c92ad-754d-4440-8999-87918e0b3f21 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:32.317144 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 211/413] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:28:32 2022] GET /identity/v3/projects/8d5de9b2c50341a8b78fb369c7765787 => generated 344 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:32.326789 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c92ad-754d-4440-8999-87918e0b3f21 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:32.448548 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd4649-000f-4c26-89f9-2bd416ac735b None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:32.485380 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2880ad05-0688-4c5a-9264-97958f3b36ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:32.486222 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2880ad05-0688-4c5a-9264-97958f3b36ca None admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:32.486558 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2880ad05-0688-4c5a-9264-97958f3b36ca None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:32.486835 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2880ad05-0688-4c5a-9264-97958f3b36ca None admin] PATH_INFO: `/v3/projects/230decc93e3a483aa3d5cc33c0f662a9/groups/9eaa3a30967242c095e3c139fb633c93/roles/50f62a33eb434682baa7529619fe14b0` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:32.489116 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2880ad05-0688-4c5a-9264-97958f3b36ca None admin] RBAC: Authorizing `identity:create_grant(project_id=230decc93e3a483aa3d5cc33c0f662a9, group_id=9eaa3a30967242c095e3c139fb633c93, role_id=50f62a33eb434682baa7529619fe14b0)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:32.490663 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2880ad05-0688-4c5a-9264-97958f3b36ca None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:32.502313 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:32.509359 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 203/414] 10.222.0.36 () {58 vars in 1442 bytes} [Thu May 19 16:28:32 2022] PUT /identity/v3/projects/230decc93e3a483aa3d5cc33c0f662a9/groups/9eaa3a30967242c095e3c139fb633c93/roles/50f62a33eb434682baa7529619fe14b0 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:28:32.623069 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c92ad-754d-4440-8999-87918e0b3f21 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:32.699850 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e772f-c4d8-441c-8f36-2c70d43f5eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:32.700761 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e772f-c4d8-441c-8f36-2c70d43f5eac None admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:32.701044 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e772f-c4d8-441c-8f36-2c70d43f5eac None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:32.701363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e772f-c4d8-441c-8f36-2c70d43f5eac None admin] PATH_INFO: `/v3/projects/8d5de9b2c50341a8b78fb369c7765787/groups/46c5c11a3ae94d158114a2ea156c9b6f/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:32.706832 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7e772f-c4d8-441c-8f36-2c70d43f5eac None admin] RBAC: Authorizing `identity:create_grant(project_id=8d5de9b2c50341a8b78fb369c7765787, group_id=46c5c11a3ae94d158114a2ea156c9b6f, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:32.708321 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7e772f-c4d8-441c-8f36-2c70d43f5eac None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:32.731513 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 212/415] 10.222.0.36 () {58 vars in 1442 bytes} [Thu May 19 16:28:32 2022] PUT /identity/v3/projects/8d5de9b2c50341a8b78fb369c7765787/groups/46c5c11a3ae94d158114a2ea156c9b6f/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:28:33.182784 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ab864-17cd-44ef-8747-99b7b30c3a19 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:33.182973 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ab864-17cd-44ef-8747-99b7b30c3a19 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:33.183263 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ab864-17cd-44ef-8747-99b7b30c3a19 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:33.184937 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 204/416] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:33.190399 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b91f1-f1be-4914-ad68-beddc37e4ed4 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:33.190698 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b91f1-f1be-4914-ad68-beddc37e4ed4 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:33.190985 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b91f1-f1be-4914-ad68-beddc37e4ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:33.210239 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-674b91f1-f1be-4914-ad68-beddc37e4ed4 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:33.211820 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-674b91f1-f1be-4914-ad68-beddc37e4ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:33.271142 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:33.298907 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 213/417] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:28:33.333651 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ab864-17cd-44ef-8747-99b7b30c3a19 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:33.478564 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ab864-17cd-44ef-8747-99b7b30c3a19 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:33.528621 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dc197d-f5a2-401d-bbed-2b51ecca0a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:33.529461 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc197d-f5a2-401d-bbed-2b51ecca0a3d None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:33.529745 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc197d-f5a2-401d-bbed-2b51ecca0a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:33.530021 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc197d-f5a2-401d-bbed-2b51ecca0a3d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:33.546622 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dc197d-f5a2-401d-bbed-2b51ecca0a3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:28:33.554479 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-12dc197d-f5a2-401d-bbed-2b51ecca0a3d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 16:28:33.554479 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:33.554479 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:33.554479 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:33.554479 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:33.554479 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:33.554479 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:33.554479 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:33.554479 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:33.554479 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:33.554479 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:33.554479 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:33.554479 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:28:33.554479 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:28:33.554479 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:28:33.554479 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:28:33.555378 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:28:33.555378 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:28:33.555378 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:28:33.555378 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:28:33.555378 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:33.555378 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:33.555378 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:33.555378 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:33.555378 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:33.555378 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:33.555378 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:33.555378 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:33.555378 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:33.555378 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:33.555378 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:33.555378 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:33.556618 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:33.556618 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:33.556618 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:33.556618 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:33.556618 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:28:33.556618 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:28:33.556618 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:28:33.556618 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:28:33.556618 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:28:33.556618 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:28:33.556618 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 16:28:33.556618 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:33.556618 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 205/418] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:28:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:33.561780 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674b91f1-f1be-4914-ad68-beddc37e4ed4 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:33.719716 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674b91f1-f1be-4914-ad68-beddc37e4ed4 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:33.769128 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d866df1a-b9cf-4a2b-a389-77145a98b989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:33.769780 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866df1a-b9cf-4a2b-a389-77145a98b989 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:33.770070 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866df1a-b9cf-4a2b-a389-77145a98b989 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:33.770357 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866df1a-b9cf-4a2b-a389-77145a98b989 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:33.771393 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d866df1a-b9cf-4a2b-a389-77145a98b989 None admin] RBAC: Adding query filter params (name=member) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:33.771708 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d866df1a-b9cf-4a2b-a389-77145a98b989 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:33.773224 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d866df1a-b9cf-4a2b-a389-77145a98b989 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:33.782904 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 214/419] 10.222.0.36 () {56 vars in 1101 bytes} [Thu May 19 16:28:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:33.788975 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dc197d-f5a2-401d-bbed-2b51ecca0a3d None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:33.906606 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:33.944148 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248e3b8-e94c-46ad-a8b2-2690d7f7a441 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:33.944784 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248e3b8-e94c-46ad-a8b2-2690d7f7a441 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:33.945423 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248e3b8-e94c-46ad-a8b2-2690d7f7a441 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:33.948443 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 215/420] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 16:28:33.952779 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de5e46b-22d7-4d3d-aab7-05fb6999e1b1 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:33.953352 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de5e46b-22d7-4d3d-aab7-05fb6999e1b1 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:33.953834 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de5e46b-22d7-4d3d-aab7-05fb6999e1b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:33.972553 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-6de5e46b-22d7-4d3d-aab7-05fb6999e1b1 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:33.974224 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-6de5e46b-22d7-4d3d-aab7-05fb6999e1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:33.980846 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dc197d-f5a2-401d-bbed-2b51ecca0a3d None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:34.019987 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:34.051444 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc5bbe6-5f19-4253-b646-d0d63fbbb25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:34.053316 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5bbe6-5f19-4253-b646-d0d63fbbb25e None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:34.056374 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5bbe6-5f19-4253-b646-d0d63fbbb25e None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:34.056890 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5bbe6-5f19-4253-b646-d0d63fbbb25e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:34.084267 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 216/421] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 16:28:34.120530 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de5e46b-22d7-4d3d-aab7-05fb6999e1b1 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:34.165188 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc5bbe6-5f19-4253-b646-d0d63fbbb25e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:34.165188 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc5bbe6-5f19-4253-b646-d0d63fbbb25e None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:34.234710 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:34.292497 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de5e46b-22d7-4d3d-aab7-05fb6999e1b1 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:34.330873 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 206/422] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:28:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 16:28:34.335653 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc5bbe6-5f19-4253-b646-d0d63fbbb25e None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:34.338820 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b47ec-ff6a-4951-84dd-68c476010343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:34.339653 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b47ec-ff6a-4951-84dd-68c476010343 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:34.339938 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b47ec-ff6a-4951-84dd-68c476010343 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:34.340214 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b47ec-ff6a-4951-84dd-68c476010343 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:34.378320 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2b47ec-ff6a-4951-84dd-68c476010343 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:28:34.386690 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-ed2b47ec-ff6a-4951-84dd-68c476010343 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 16:28:34.386690 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:34.386690 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:34.386690 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:34.386690 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:34.386690 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:34.386690 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:34.386690 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:34.386690 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:34.386690 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:34.386690 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:34.386690 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:34.386690 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:28:34.386690 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:28:34.386690 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:28:34.386690 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:28:34.387639 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:28:34.387639 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:28:34.387639 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:28:34.387639 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:28:34.387639 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:34.387639 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:34.387639 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:34.387639 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:34.387639 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:34.387639 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:34.387639 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:34.387639 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:28:34.387639 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:34.387639 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:34.387639 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:34.387639 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:34.388459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:34.388459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:34.388459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:34.388459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:34.388459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:28:34.388459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:28:34.388459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:28:34.388459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:28:34.388459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:28:34.388459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:28:34.388459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 16:28:34.388459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:28:34.388459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 217/423] 10.222.0.36 () {56 vars in 1114 bytes} [Thu May 19 16:28:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 16:28:34.394888 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b47ec-ff6a-4951-84dd-68c476010343 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:34.459504 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc5bbe6-5f19-4253-b646-d0d63fbbb25e None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:34.494473 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:34.503165 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eec5ba-69e2-4b5d-97b3-e206fcf68da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:34.504295 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eec5ba-69e2-4b5d-97b3-e206fcf68da9 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:34.504796 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eec5ba-69e2-4b5d-97b3-e206fcf68da9 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:34.505256 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eec5ba-69e2-4b5d-97b3-e206fcf68da9 None admin] PATH_INFO: `/v3/projects/d4a456cd7f83414a895ab7982fec2a68` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:34.507071 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eec5ba-69e2-4b5d-97b3-e206fcf68da9 None admin] RBAC: Authorizing `identity:get_project(project_id=d4a456cd7f83414a895ab7982fec2a68)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:34.508691 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eec5ba-69e2-4b5d-97b3-e206fcf68da9 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:34.510937 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 207/424] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:28:34 2022] GET /identity/v3/projects/d4a456cd7f83414a895ab7982fec2a68 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:28:34.517101 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eec5ba-69e2-4b5d-97b3-e206fcf68da9 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:34.551323 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:34.632515 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b47ec-ff6a-4951-84dd-68c476010343 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:34.673775 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118eaf3d-79c3-4e8a-862d-f182024caf86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:34.674730 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118eaf3d-79c3-4e8a-862d-f182024caf86 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:34.675208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118eaf3d-79c3-4e8a-862d-f182024caf86 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:34.675682 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118eaf3d-79c3-4e8a-862d-f182024caf86 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:34.676629 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118eaf3d-79c3-4e8a-862d-f182024caf86 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:34.677111 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118eaf3d-79c3-4e8a-862d-f182024caf86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:34.678788 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118eaf3d-79c3-4e8a-862d-f182024caf86 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:34.688924 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 218/425] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:28:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:34.697213 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118eaf3d-79c3-4e8a-862d-f182024caf86 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:34.698287 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eec5ba-69e2-4b5d-97b3-e206fcf68da9 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:34.739379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a52687-1373-4fbb-8699-548eb4aca520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:34.740276 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a52687-1373-4fbb-8699-548eb4aca520 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:34.740563 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a52687-1373-4fbb-8699-548eb4aca520 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:34.740851 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a52687-1373-4fbb-8699-548eb4aca520 None admin] PATH_INFO: `/v3/groups/9eaa3a30967242c095e3c139fb633c93` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:34.745071 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a52687-1373-4fbb-8699-548eb4aca520 None admin] RBAC: Authorizing `identity:get_group(group_id=9eaa3a30967242c095e3c139fb633c93)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:34.746594 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a52687-1373-4fbb-8699-548eb4aca520 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:34.748280 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 208/426] 10.222.0.36 () {56 vars in 1180 bytes} [Thu May 19 16:28:34 2022] GET /identity/v3/groups/9eaa3a30967242c095e3c139fb633c93 => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:34.762412 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a52687-1373-4fbb-8699-548eb4aca520 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:34.877552 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:34.889521 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a52687-1373-4fbb-8699-548eb4aca520 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:34.900407 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:34.913397 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118eaf3d-79c3-4e8a-862d-f182024caf86 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:34.939056 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f8bdac-db08-4222-9a7d-641954eca072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:34.940203 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f8bdac-db08-4222-9a7d-641954eca072 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:34.940736 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f8bdac-db08-4222-9a7d-641954eca072 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:34.941276 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f8bdac-db08-4222-9a7d-641954eca072 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:34.942370 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f8bdac-db08-4222-9a7d-641954eca072 None admin] RBAC: Adding query filter params (group.id=9eaa3a30967242c095e3c139fb633c93, role.id=0e98fd94a6d24e46b9fe09a22a7df04a, scope.project.id=d4a456cd7f83414a895ab7982fec2a68) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:34.942906 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f8bdac-db08-4222-9a7d-641954eca072 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:34.944593 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f8bdac-db08-4222-9a7d-641954eca072 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:34.957123 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 209/427] 10.222.0.36 () {56 vars in 1376 bytes} [Thu May 19 16:28:34 2022] GET /identity/v3/role_assignments?group.id=9eaa3a30967242c095e3c139fb633c93&scope.project.id=d4a456cd7f83414a895ab7982fec2a68&role.id=0e98fd94a6d24e46b9fe09a22a7df04a => generated 611 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:28:34.961560 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d094bb-002a-4fba-87ec-1ba5538b0915 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:34.962613 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d094bb-002a-4fba-87ec-1ba5538b0915 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:34.963008 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d094bb-002a-4fba-87ec-1ba5538b0915 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:34.963510 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f731842-bc07-4b41-b121-814aaa1a0e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:34.964359 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f731842-bc07-4b41-b121-814aaa1a0e44 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:34.964857 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f731842-bc07-4b41-b121-814aaa1a0e44 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:34.969198 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 210/428] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:34.969739 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f731842-bc07-4b41-b121-814aaa1a0e44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:34.972534 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be451bc6-1704-4a43-b302-41a57758e3a4 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:34.972827 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be451bc6-1704-4a43-b302-41a57758e3a4 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:34.973245 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be451bc6-1704-4a43-b302-41a57758e3a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:35.008006 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-be451bc6-1704-4a43-b302-41a57758e3a4 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:35.010283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-be451bc6-1704-4a43-b302-41a57758e3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:35.124280 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f731842-bc07-4b41-b121-814aaa1a0e44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:35.126316 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f731842-bc07-4b41-b121-814aaa1a0e44 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:35.158894 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 211/429] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:28:35.197138 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be451bc6-1704-4a43-b302-41a57758e3a4 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:35.228772 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:35.241121 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 219/430] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:28:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 16:28:35.252593 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f731842-bc07-4b41-b121-814aaa1a0e44 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:35.321345 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:35.337531 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be451bc6-1704-4a43-b302-41a57758e3a4 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:35.396088 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49fd8cc-0270-4df6-ab7a-cab744beccd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:35.397067 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f731842-bc07-4b41-b121-814aaa1a0e44 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:35.397885 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49fd8cc-0270-4df6-ab7a-cab744beccd5 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:35.398367 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49fd8cc-0270-4df6-ab7a-cab744beccd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:35.398840 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49fd8cc-0270-4df6-ab7a-cab744beccd5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:35.417013 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49fd8cc-0270-4df6-ab7a-cab744beccd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:28:35.427435 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-f49fd8cc-0270-4df6-ab7a-cab744beccd5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 16:28:35.427435 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:35.427435 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:35.427435 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:35.427435 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:35.427435 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:35.427435 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:35.427435 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:35.427435 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:35.427435 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:35.427435 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:35.427435 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:35.427435 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:28:35.427435 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:28:35.427435 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:28:35.427435 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:28:35.428386 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:28:35.428386 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:28:35.428386 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:28:35.428386 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:28:35.428386 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:35.428386 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:35.428386 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:35.428386 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:35.428386 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:35.428386 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:35.428386 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:35.428386 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:35.428386 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:35.428386 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:35.428386 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:35.428386 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:35.429228 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:35.429228 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:35.429228 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:35.429228 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:35.429228 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:28:35.429228 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:28:35.429228 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:28:35.429228 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:28:35.429228 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:28:35.429228 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:28:35.429228 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 16:28:35.429228 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:35.437393 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 212/431] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:28:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 16:28:35.441708 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49fd8cc-0270-4df6-ab7a-cab744beccd5 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:35.463584 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6d7af4-5e78-4401-ae49-4a30506ad534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:35.464474 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d7af4-5e78-4401-ae49-4a30506ad534 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:35.464768 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d7af4-5e78-4401-ae49-4a30506ad534 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:35.465054 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d7af4-5e78-4401-ae49-4a30506ad534 None admin] PATH_INFO: `/v3/projects/d4a456cd7f83414a895ab7982fec2a68` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:35.466345 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6d7af4-5e78-4401-ae49-4a30506ad534 None admin] RBAC: Authorizing `identity:get_project(project_id=d4a456cd7f83414a895ab7982fec2a68)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:35.467937 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6d7af4-5e78-4401-ae49-4a30506ad534 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:35.470023 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 220/432] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:28:35 2022] GET /identity/v3/projects/d4a456cd7f83414a895ab7982fec2a68 => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:35.471906 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6eeafd-a17f-4b08-a540-a65375d9c2f7 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:35.472189 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6eeafd-a17f-4b08-a540-a65375d9c2f7 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:35.472473 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6eeafd-a17f-4b08-a540-a65375d9c2f7 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:35.473993 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 221/433] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:35.478052 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6eeafd-a17f-4b08-a540-a65375d9c2f7 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:35.566082 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:35.577196 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49fd8cc-0270-4df6-ab7a-cab744beccd5 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:35.614735 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6eeafd-a17f-4b08-a540-a65375d9c2f7 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:35.620324 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e757191-8e36-4c70-9704-02e332a569da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:35.621102 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e757191-8e36-4c70-9704-02e332a569da None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:35.621481 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e757191-8e36-4c70-9704-02e332a569da None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:35.621762 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e757191-8e36-4c70-9704-02e332a569da None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:35.622534 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e757191-8e36-4c70-9704-02e332a569da None admin] RBAC: Adding query filter params (name=member) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:35.622837 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e757191-8e36-4c70-9704-02e332a569da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:35.624380 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e757191-8e36-4c70-9704-02e332a569da None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:35.634398 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 213/434] 10.222.0.36 () {56 vars in 1101 bytes} [Thu May 19 16:28:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:35.635555 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece4234-9582-441e-a530-98f3924674ee None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:35.635848 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece4234-9582-441e-a530-98f3924674ee None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:35.636190 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece4234-9582-441e-a530-98f3924674ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:35.654127 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-3ece4234-9582-441e-a530-98f3924674ee None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:35.655301 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-3ece4234-9582-441e-a530-98f3924674ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:35.663549 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:35.665334 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad07580a-6eb9-4268-bba8-baba7be15b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:35.666188 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad07580a-6eb9-4268-bba8-baba7be15b79 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:35.666478 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad07580a-6eb9-4268-bba8-baba7be15b79 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:35.666755 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad07580a-6eb9-4268-bba8-baba7be15b79 None admin] PATH_INFO: `/v3/groups/9eaa3a30967242c095e3c139fb633c93` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:35.668016 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad07580a-6eb9-4268-bba8-baba7be15b79 None admin] RBAC: Authorizing `identity:get_group(group_id=9eaa3a30967242c095e3c139fb633c93)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:35.669488 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad07580a-6eb9-4268-bba8-baba7be15b79 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:35.671143 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 222/435] 10.222.0.36 () {56 vars in 1180 bytes} [Thu May 19 16:28:35 2022] GET /identity/v3/groups/9eaa3a30967242c095e3c139fb633c93 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:35.672341 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c8298-5cbc-4240-9776-018a134d56cd None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:35.672633 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c8298-5cbc-4240-9776-018a134d56cd None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:35.672915 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c8298-5cbc-4240-9776-018a134d56cd None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:35.678215 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 223/436] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:35.678215 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c8298-5cbc-4240-9776-018a134d56cd None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:35.750885 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 214/437] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:28:35.753602 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece4234-9582-441e-a530-98f3924674ee None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:35.788031 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c8298-5cbc-4240-9776-018a134d56cd None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:35.818422 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:35.825163 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8d08b9-fc1f-429b-a18d-873391c02258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:35.826223 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d08b9-fc1f-429b-a18d-873391c02258 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:35.826635 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d08b9-fc1f-429b-a18d-873391c02258 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:35.827039 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d08b9-fc1f-429b-a18d-873391c02258 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:35.861706 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece4234-9582-441e-a530-98f3924674ee None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:35.900180 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fba5fd-4cd8-4acd-9f42-ccd27e5a6db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:35.900851 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fba5fd-4cd8-4acd-9f42-ccd27e5a6db1 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:35.901135 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fba5fd-4cd8-4acd-9f42-ccd27e5a6db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:35.901443 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fba5fd-4cd8-4acd-9f42-ccd27e5a6db1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:35.902285 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fba5fd-4cd8-4acd-9f42-ccd27e5a6db1 None admin] RBAC: Adding query filter params (group.id=9eaa3a30967242c095e3c139fb633c93, role.id=50f62a33eb434682baa7529619fe14b0, scope.project.id=d4a456cd7f83414a895ab7982fec2a68) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:35.902598 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fba5fd-4cd8-4acd-9f42-ccd27e5a6db1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:35.902740 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8d08b9-fc1f-429b-a18d-873391c02258 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:35.904039 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fba5fd-4cd8-4acd-9f42-ccd27e5a6db1 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:35.904200 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8d08b9-fc1f-429b-a18d-873391c02258 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:35.910969 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 215/438] 10.222.0.36 () {56 vars in 1376 bytes} [Thu May 19 16:28:35 2022] GET /identity/v3/role_assignments?group.id=9eaa3a30967242c095e3c139fb633c93&scope.project.id=d4a456cd7f83414a895ab7982fec2a68&role.id=50f62a33eb434682baa7529619fe14b0 => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:35.912343 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63919ba-31c6-4cac-9413-bfdb0a25ea01 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:35.912641 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63919ba-31c6-4cac-9413-bfdb0a25ea01 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:35.913661 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63919ba-31c6-4cac-9413-bfdb0a25ea01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:35.940072 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:35.946085 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-c63919ba-31c6-4cac-9413-bfdb0a25ea01 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:35.947416 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-c63919ba-31c6-4cac-9413-bfdb0a25ea01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:36.024787 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 224/439] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:28:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 16:28:36.026065 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8d08b9-fc1f-429b-a18d-873391c02258 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:36.026419 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 216/440] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:28:36.030675 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63919ba-31c6-4cac-9413-bfdb0a25ea01 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:36.067171 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:36.168751 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8d08b9-fc1f-429b-a18d-873391c02258 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:36.171350 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63919ba-31c6-4cac-9413-bfdb0a25ea01 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:36.213259 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06826ae2-10fa-4571-a5d6-5e86883383b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:36.214149 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06826ae2-10fa-4571-a5d6-5e86883383b8 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:36.214503 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06826ae2-10fa-4571-a5d6-5e86883383b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:36.214877 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06826ae2-10fa-4571-a5d6-5e86883383b8 None admin] PATH_INFO: `/v3/projects/230decc93e3a483aa3d5cc33c0f662a9` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:36.216738 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06826ae2-10fa-4571-a5d6-5e86883383b8 None admin] RBAC: Authorizing `identity:get_project(project_id=230decc93e3a483aa3d5cc33c0f662a9)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:36.218288 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06826ae2-10fa-4571-a5d6-5e86883383b8 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:36.218437 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a100ad47-883e-4ce1-a896-925ba7a3c9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:36.219358 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100ad47-883e-4ce1-a896-925ba7a3c9e7 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:36.219637 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100ad47-883e-4ce1-a896-925ba7a3c9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:36.220168 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100ad47-883e-4ce1-a896-925ba7a3c9e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:36.220818 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 217/441] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:28:36 2022] GET /identity/v3/projects/230decc93e3a483aa3d5cc33c0f662a9 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:36.222560 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06826ae2-10fa-4571-a5d6-5e86883383b8 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:36.227193 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:36.233114 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a100ad47-883e-4ce1-a896-925ba7a3c9e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:28:36.242088 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-a100ad47-883e-4ce1-a896-925ba7a3c9e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:28:36.242088 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:36.242088 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:36.242088 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:36.242088 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:36.242088 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:36.242088 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:36.242088 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:36.242088 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:36.242088 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:36.242088 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:36.242088 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:36.242088 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:28:36.242088 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:28:36.242088 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:28:36.242088 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:28:36.243051 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:28:36.243051 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:28:36.243051 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:28:36.243051 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:28:36.243051 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:36.243051 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:36.243051 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:36.243051 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:36.243051 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:36.243051 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:36.243051 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:36.243051 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:28:36.243051 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:36.243051 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:36.243051 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:36.243051 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:36.243866 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:36.243866 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:36.243866 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:36.243866 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:36.243866 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:28:36.243866 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:28:36.243866 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:28:36.243866 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:28:36.243866 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:28:36.243866 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:28:36.243866 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:28:36.243866 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:28:36.243866 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 225/442] 10.222.0.36 () {56 vars in 1096 bytes} [Thu May 19 16:28:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:36.244526 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a100ad47-883e-4ce1-a896-925ba7a3c9e7 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:36.347414 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06826ae2-10fa-4571-a5d6-5e86883383b8 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:36.353397 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:36.363512 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a100ad47-883e-4ce1-a896-925ba7a3c9e7 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:36.383502 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4687f068-6064-4994-ac32-883c2776e0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:36.384331 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4687f068-6064-4994-ac32-883c2776e0ee None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:36.384613 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4687f068-6064-4994-ac32-883c2776e0ee None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:36.384888 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4687f068-6064-4994-ac32-883c2776e0ee None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:36.395527 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4687f068-6064-4994-ac32-883c2776e0ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:28:36.399138 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b06e1b-38bd-4387-8a47-3ec83053e5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:36.399996 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b06e1b-38bd-4387-8a47-3ec83053e5c3 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:36.400281 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b06e1b-38bd-4387-8a47-3ec83053e5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:36.400551 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b06e1b-38bd-4387-8a47-3ec83053e5c3 None admin] PATH_INFO: `/v3/groups/9eaa3a30967242c095e3c139fb633c93` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:36.401767 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b06e1b-38bd-4387-8a47-3ec83053e5c3 None admin] RBAC: Authorizing `identity:get_group(group_id=9eaa3a30967242c095e3c139fb633c93)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:36.403236 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b06e1b-38bd-4387-8a47-3ec83053e5c3 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:36.403850 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-4687f068-6064-4994-ac32-883c2776e0ee None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 16:28:36.403850 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:36.403850 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:36.403850 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:36.403850 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:36.403850 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:36.403850 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:36.403850 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:36.403850 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:36.403850 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:36.403850 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:36.403850 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:36.403850 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:28:36.403850 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:28:36.403850 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:28:36.403850 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:28:36.404794 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:28:36.404794 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:28:36.404794 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:28:36.404794 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:28:36.404794 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:36.404794 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:36.404794 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:36.404794 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:36.404794 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:36.404794 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:36.404794 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:36.404794 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:36.404794 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:36.404794 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:36.404794 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:36.404794 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:36.406213 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:36.406213 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:36.406213 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:36.406213 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:36.406213 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:28:36.406213 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:28:36.406213 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:28:36.406213 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:28:36.406213 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:28:36.406213 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:28:36.406213 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 16:28:36.406213 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:36.406839 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 226/443] 10.222.0.36 () {56 vars in 1180 bytes} [Thu May 19 16:28:36 2022] GET /identity/v3/groups/9eaa3a30967242c095e3c139fb633c93 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:36.406909 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 218/444] 10.222.0.36 () {56 vars in 1114 bytes} [Thu May 19 16:28:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:36.409829 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b06e1b-38bd-4387-8a47-3ec83053e5c3 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:36.411092 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4687f068-6064-4994-ac32-883c2776e0ee None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:36.476535 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:36.512021 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b06e1b-38bd-4387-8a47-3ec83053e5c3 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:36.514462 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4687f068-6064-4994-ac32-883c2776e0ee None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:36.545936 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9656cf4-aba7-4df9-b318-65c04d9db22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:36.546608 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9656cf4-aba7-4df9-b318-65c04d9db22b None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:36.546892 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9656cf4-aba7-4df9-b318-65c04d9db22b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:36.547167 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9656cf4-aba7-4df9-b318-65c04d9db22b None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:36.547918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9656cf4-aba7-4df9-b318-65c04d9db22b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:36.548222 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9656cf4-aba7-4df9-b318-65c04d9db22b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:36.549013 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2883ea-f9d6-4de3-b8d8-52f3efdc368f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:36.549701 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2883ea-f9d6-4de3-b8d8-52f3efdc368f None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:36.549993 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2883ea-f9d6-4de3-b8d8-52f3efdc368f None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:36.550271 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2883ea-f9d6-4de3-b8d8-52f3efdc368f None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:36.550387 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9656cf4-aba7-4df9-b318-65c04d9db22b None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:36.551022 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2883ea-f9d6-4de3-b8d8-52f3efdc368f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:36.551333 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2883ea-f9d6-4de3-b8d8-52f3efdc368f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:36.552688 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2883ea-f9d6-4de3-b8d8-52f3efdc368f None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:36.558636 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 227/445] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:28:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:36.559445 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9656cf4-aba7-4df9-b318-65c04d9db22b None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:36.565126 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 219/446] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:28:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:36.565126 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2883ea-f9d6-4de3-b8d8-52f3efdc368f None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:36.581439 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:36.663008 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9656cf4-aba7-4df9-b318-65c04d9db22b None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:36.666442 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2883ea-f9d6-4de3-b8d8-52f3efdc368f None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:36.694397 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:36.696751 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2435b696-1a1e-476e-a7c4-9541b85ea1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:36.697473 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435b696-1a1e-476e-a7c4-9541b85ea1eb None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:36.697856 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435b696-1a1e-476e-a7c4-9541b85ea1eb None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:36.698109 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435b696-1a1e-476e-a7c4-9541b85ea1eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:36.698878 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2435b696-1a1e-476e-a7c4-9541b85ea1eb None admin] RBAC: Adding query filter params (group.id=9eaa3a30967242c095e3c139fb633c93, role.id=0e98fd94a6d24e46b9fe09a22a7df04a, scope.project.id=230decc93e3a483aa3d5cc33c0f662a9) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:36.699230 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2435b696-1a1e-476e-a7c4-9541b85ea1eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:36.700659 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2435b696-1a1e-476e-a7c4-9541b85ea1eb None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:36.707400 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 228/447] 10.222.0.36 () {56 vars in 1376 bytes} [Thu May 19 16:28:36 2022] GET /identity/v3/role_assignments?group.id=9eaa3a30967242c095e3c139fb633c93&scope.project.id=230decc93e3a483aa3d5cc33c0f662a9&role.id=0e98fd94a6d24e46b9fe09a22a7df04a => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:36.708187 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2435b696-1a1e-476e-a7c4-9541b85ea1eb None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:36.708710 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f8b65f-28d5-41bd-b833-5c6aec0aea03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:36.709748 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f8b65f-28d5-41bd-b833-5c6aec0aea03 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:36.710235 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f8b65f-28d5-41bd-b833-5c6aec0aea03 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:36.710636 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f8b65f-28d5-41bd-b833-5c6aec0aea03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:36.800104 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f8b65f-28d5-41bd-b833-5c6aec0aea03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:36.801649 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f8b65f-28d5-41bd-b833-5c6aec0aea03 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:36.819652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:36.839675 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2435b696-1a1e-476e-a7c4-9541b85ea1eb None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:36.878227 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9708719-307d-45a8-aaac-c9143d9189de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:36.878905 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9708719-307d-45a8-aaac-c9143d9189de None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:36.879191 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9708719-307d-45a8-aaac-c9143d9189de None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:36.879504 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9708719-307d-45a8-aaac-c9143d9189de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:36.932970 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 220/448] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:28:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 16:28:36.938095 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f8b65f-28d5-41bd-b833-5c6aec0aea03 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:36.963474 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9708719-307d-45a8-aaac-c9143d9189de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:36.964982 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9708719-307d-45a8-aaac-c9143d9189de None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:36.972544 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:37.061501 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:37.073805 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f8b65f-28d5-41bd-b833-5c6aec0aea03 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:37.080901 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 229/449] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:28:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 16:28:37.086707 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9708719-307d-45a8-aaac-c9143d9189de None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:37.114702 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c500a87f-e699-4714-ab08-7650dee88360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:37.115567 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c500a87f-e699-4714-ab08-7650dee88360 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:37.115852 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c500a87f-e699-4714-ab08-7650dee88360 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:37.116135 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c500a87f-e699-4714-ab08-7650dee88360 None admin] PATH_INFO: `/v3/projects/8d5de9b2c50341a8b78fb369c7765787` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:37.117837 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c500a87f-e699-4714-ab08-7650dee88360 None admin] RBAC: Authorizing `identity:get_project(project_id=8d5de9b2c50341a8b78fb369c7765787)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:37.119308 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c500a87f-e699-4714-ab08-7650dee88360 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:37.121355 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 221/450] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:28:36 2022] GET /identity/v3/projects/8d5de9b2c50341a8b78fb369c7765787 => generated 344 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:28:37.127132 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c500a87f-e699-4714-ab08-7650dee88360 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:37.191046 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9708719-307d-45a8-aaac-c9143d9189de None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:37.221998 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:37.226900 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecead105-c69f-44a3-99c2-f40a344f1915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:37.227772 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecead105-c69f-44a3-99c2-f40a344f1915 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:37.228053 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecead105-c69f-44a3-99c2-f40a344f1915 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:37.228331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecead105-c69f-44a3-99c2-f40a344f1915 None admin] PATH_INFO: `/v3/projects/230decc93e3a483aa3d5cc33c0f662a9` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:37.229427 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c500a87f-e699-4714-ab08-7650dee88360 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:37.229579 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecead105-c69f-44a3-99c2-f40a344f1915 None admin] RBAC: Authorizing `identity:get_project(project_id=230decc93e3a483aa3d5cc33c0f662a9)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:37.231054 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecead105-c69f-44a3-99c2-f40a344f1915 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:37.232779 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 230/451] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:28:37 2022] GET /identity/v3/projects/230decc93e3a483aa3d5cc33c0f662a9 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:28:37.238580 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecead105-c69f-44a3-99c2-f40a344f1915 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:37.265844 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e8bb84-0802-46fe-a6f5-7b6c35f67cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:37.266706 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8bb84-0802-46fe-a6f5-7b6c35f67cd5 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:37.266990 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8bb84-0802-46fe-a6f5-7b6c35f67cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:37.267848 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8bb84-0802-46fe-a6f5-7b6c35f67cd5 None admin] PATH_INFO: `/v3/groups/46c5c11a3ae94d158114a2ea156c9b6f` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:37.269028 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e8bb84-0802-46fe-a6f5-7b6c35f67cd5 None admin] RBAC: Authorizing `identity:get_group(group_id=46c5c11a3ae94d158114a2ea156c9b6f)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:37.270794 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e8bb84-0802-46fe-a6f5-7b6c35f67cd5 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:37.272431 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 222/452] 10.222.0.36 () {56 vars in 1180 bytes} [Thu May 19 16:28:37 2022] GET /identity/v3/groups/46c5c11a3ae94d158114a2ea156c9b6f => generated 229 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:28:37.280298 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e8bb84-0802-46fe-a6f5-7b6c35f67cd5 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:37.293945 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:37.341122 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecead105-c69f-44a3-99c2-f40a344f1915 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:37.375254 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eebc02-5d18-449e-84d2-cee6a454f821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:37.376123 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eebc02-5d18-449e-84d2-cee6a454f821 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:37.376408 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eebc02-5d18-449e-84d2-cee6a454f821 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:37.377267 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eebc02-5d18-449e-84d2-cee6a454f821 None admin] PATH_INFO: `/v3/groups/9eaa3a30967242c095e3c139fb633c93` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:37.378453 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eebc02-5d18-449e-84d2-cee6a454f821 None admin] RBAC: Authorizing `identity:get_group(group_id=9eaa3a30967242c095e3c139fb633c93)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:37.379914 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eebc02-5d18-449e-84d2-cee6a454f821 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:37.380067 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e8bb84-0802-46fe-a6f5-7b6c35f67cd5 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:37.381932 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 231/453] 10.222.0.36 () {56 vars in 1180 bytes} [Thu May 19 16:28:37 2022] GET /identity/v3/groups/9eaa3a30967242c095e3c139fb633c93 => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:28:37.387265 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eebc02-5d18-449e-84d2-cee6a454f821 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:37.413887 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9912fd1e-e3eb-423f-bfaa-53d10ac4451b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:37.414551 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912fd1e-e3eb-423f-bfaa-53d10ac4451b None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:37.414833 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912fd1e-e3eb-423f-bfaa-53d10ac4451b None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:37.415112 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912fd1e-e3eb-423f-bfaa-53d10ac4451b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:37.415963 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9912fd1e-e3eb-423f-bfaa-53d10ac4451b None admin] RBAC: Adding query filter params (group.id=46c5c11a3ae94d158114a2ea156c9b6f, role.id=e2915fe109244a07a9be96bd2b219fdd, scope.project.id=8d5de9b2c50341a8b78fb369c7765787) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:37.416277 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9912fd1e-e3eb-423f-bfaa-53d10ac4451b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:37.417752 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9912fd1e-e3eb-423f-bfaa-53d10ac4451b None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:37.424131 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 223/454] 10.222.0.36 () {56 vars in 1376 bytes} [Thu May 19 16:28:37 2022] GET /identity/v3/role_assignments?group.id=46c5c11a3ae94d158114a2ea156c9b6f&scope.project.id=8d5de9b2c50341a8b78fb369c7765787&role.id=e2915fe109244a07a9be96bd2b219fdd => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:28:37.511836 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eebc02-5d18-449e-84d2-cee6a454f821 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:37.533997 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:37.550524 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41861899-3784-43c9-a280-120de4bb4520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:37.551199 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41861899-3784-43c9-a280-120de4bb4520 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:37.551486 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41861899-3784-43c9-a280-120de4bb4520 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:37.551766 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41861899-3784-43c9-a280-120de4bb4520 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:37.552625 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41861899-3784-43c9-a280-120de4bb4520 None admin] RBAC: Adding query filter params (group.id=9eaa3a30967242c095e3c139fb633c93, role.id=50f62a33eb434682baa7529619fe14b0, scope.project.id=230decc93e3a483aa3d5cc33c0f662a9) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:37.552941 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41861899-3784-43c9-a280-120de4bb4520 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:37.554480 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41861899-3784-43c9-a280-120de4bb4520 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:37.561317 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 232/455] 10.222.0.36 () {56 vars in 1376 bytes} [Thu May 19 16:28:37 2022] GET /identity/v3/role_assignments?group.id=9eaa3a30967242c095e3c139fb633c93&scope.project.id=230decc93e3a483aa3d5cc33c0f662a9&role.id=50f62a33eb434682baa7529619fe14b0 => generated 611 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:28:40.496283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d9a35-3a72-49cf-adb5-07b1a7547271 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:40.497387 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d9a35-3a72-49cf-adb5-07b1a7547271 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:40.497891 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d9a35-3a72-49cf-adb5-07b1a7547271 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:40.500143 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 224/456] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:40.506823 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e781fdab-a1a5-40db-b901-1db62fe91833 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:40.507819 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e781fdab-a1a5-40db-b901-1db62fe91833 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:40.508415 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e781fdab-a1a5-40db-b901-1db62fe91833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:40.534256 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-e781fdab-a1a5-40db-b901-1db62fe91833 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:40.535851 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e781fdab-a1a5-40db-b901-1db62fe91833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:40.622694 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 233/457] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:28:40.662099 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725d9a35-3a72-49cf-adb5-07b1a7547271 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:40.691867 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:40.698397 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f35ea-accb-4757-a3a0-ec467f450957 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:40.699019 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f35ea-accb-4757-a3a0-ec467f450957 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:40.699527 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f35ea-accb-4757-a3a0-ec467f450957 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:40.701689 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 234/458] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:40.707223 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d1695-0dcf-4f5c-b7ab-940d0becbcda None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:40.707833 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d1695-0dcf-4f5c-b7ab-940d0becbcda None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:40.708308 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d1695-0dcf-4f5c-b7ab-940d0becbcda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:40.735061 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-ce7d1695-0dcf-4f5c-b7ab-940d0becbcda None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:40.737468 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-ce7d1695-0dcf-4f5c-b7ab-940d0becbcda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:40.857218 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 235/459] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 16:28:40.868345 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725d9a35-3a72-49cf-adb5-07b1a7547271 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:40.884125 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7d1695-0dcf-4f5c-b7ab-940d0becbcda None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:40.926903 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da7392-3466-479e-a2f0-baf1f5aa796a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:40.928175 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da7392-3466-479e-a2f0-baf1f5aa796a None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:40.928667 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da7392-3466-479e-a2f0-baf1f5aa796a None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:40.929167 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da7392-3466-479e-a2f0-baf1f5aa796a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:40.943071 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67da7392-3466-479e-a2f0-baf1f5aa796a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:40.944967 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67da7392-3466-479e-a2f0-baf1f5aa796a None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:40.985850 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-67da7392-3466-479e-a2f0-baf1f5aa796a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:40.985850 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:40.985850 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:40.985850 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:40.985850 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:40.985850 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:40.985850 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:40.985850 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:40.985850 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:40.985850 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:40.985850 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:40.985850 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:40.985850 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:28:40.985850 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:28:40.985850 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:28:40.986806 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:28:40.986806 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:40.986806 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:40.986806 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:40.986806 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:40.986806 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:40.986806 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:40.986806 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:40.986806 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:40.986806 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:40.986806 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:40.986806 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:40.986806 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:40.986806 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:40.986806 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:40.986806 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:40.986806 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:40.987652 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:28:40.987652 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:28:40.987652 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:40.987652 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:40.990189 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 225/460] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:28:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:40.996815 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da7392-3466-479e-a2f0-baf1f5aa796a None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:41.009572 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:41.041301 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7d1695-0dcf-4f5c-b7ab-940d0becbcda None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:41.078312 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5519bf-7f78-4f42-bdf0-8d5631ce81b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:41.079124 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5519bf-7f78-4f42-bdf0-8d5631ce81b1 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:41.079468 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5519bf-7f78-4f42-bdf0-8d5631ce81b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:41.080276 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5519bf-7f78-4f42-bdf0-8d5631ce81b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:41.091993 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5519bf-7f78-4f42-bdf0-8d5631ce81b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:41.093644 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5519bf-7f78-4f42-bdf0-8d5631ce81b1 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:41.115363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-aa5519bf-7f78-4f42-bdf0-8d5631ce81b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:41.115363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:41.115363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:41.115363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:41.115363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:41.115363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:41.115363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:41.115363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:41.115363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:41.115363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:41.115363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:41.115363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:41.115363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:28:41.115363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:28:41.115363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:28:41.116400 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:28:41.116400 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:41.116400 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:41.116400 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:41.116400 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:41.116400 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:41.116400 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:41.116400 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:41.116400 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:28:41.116400 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:41.116400 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:41.116400 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:41.116400 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:41.116400 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:41.116400 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:41.116400 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:41.116400 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:41.117493 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:28:41.117493 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:28:41.117493 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:41.117493 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:28:41.117493 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 236/461] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:28:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 16:28:41.117772 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed2d3f-aa6e-4937-9dae-3e7aee3585a6 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:41.117846 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed2d3f-aa6e-4937-9dae-3e7aee3585a6 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:41.118110 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed2d3f-aa6e-4937-9dae-3e7aee3585a6 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:41.123600 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 237/462] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:41.123600 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ed2d3f-aa6e-4937-9dae-3e7aee3585a6 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:41.131465 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:41.165867 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da7392-3466-479e-a2f0-baf1f5aa796a None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:41.207207 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e348c6-b792-4834-9c84-bd74842af6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:41.207889 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e348c6-b792-4834-9c84-bd74842af6ae None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:41.208185 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e348c6-b792-4834-9c84-bd74842af6ae None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:41.208467 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e348c6-b792-4834-9c84-bd74842af6ae None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:41.209240 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e348c6-b792-4834-9c84-bd74842af6ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:41.209568 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e348c6-b792-4834-9c84-bd74842af6ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:41.210966 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e348c6-b792-4834-9c84-bd74842af6ae None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:41.229390 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 226/463] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:28:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:41.229390 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba5f6d-6460-47a2-96a3-b2f8b3019cf0 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:41.229390 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba5f6d-6460-47a2-96a3-b2f8b3019cf0 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:41.229390 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba5f6d-6460-47a2-96a3-b2f8b3019cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:41.234012 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:41.241894 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ed2d3f-aa6e-4937-9dae-3e7aee3585a6 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:41.248554 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-a2ba5f6d-6460-47a2-96a3-b2f8b3019cf0 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:41.249811 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-a2ba5f6d-6460-47a2-96a3-b2f8b3019cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:41.283252 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e0e5e-925c-44f8-8afe-56ee183b581c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:41.283929 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e0e5e-925c-44f8-8afe-56ee183b581c None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:41.284218 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e0e5e-925c-44f8-8afe-56ee183b581c None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:41.284501 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e0e5e-925c-44f8-8afe-56ee183b581c None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:41.285279 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239e0e5e-925c-44f8-8afe-56ee183b581c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:41.285604 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239e0e5e-925c-44f8-8afe-56ee183b581c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:41.287034 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239e0e5e-925c-44f8-8afe-56ee183b581c None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:41.304877 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 238/464] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:28:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:41.306441 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6044589-6d2c-4df7-883d-cc2b2e83be81 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:41.306745 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6044589-6d2c-4df7-883d-cc2b2e83be81 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:41.307035 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6044589-6d2c-4df7-883d-cc2b2e83be81 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:41.308868 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 239/465] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:41.309516 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6044589-6d2c-4df7-883d-cc2b2e83be81 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:41.326986 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 227/466] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:28:41.338118 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba5f6d-6460-47a2-96a3-b2f8b3019cf0 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:41.416608 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:41.428680 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6044589-6d2c-4df7-883d-cc2b2e83be81 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:41.455958 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba5f6d-6460-47a2-96a3-b2f8b3019cf0 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:41.468975 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26b5905-ec81-4061-bc94-8294458854c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:41.470630 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b5905-ec81-4061-bc94-8294458854c6 None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:41.471419 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b5905-ec81-4061-bc94-8294458854c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:41.472160 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b5905-ec81-4061-bc94-8294458854c6 None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:41.473042 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26b5905-ec81-4061-bc94-8294458854c6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:41.475107 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26b5905-ec81-4061-bc94-8294458854c6 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:41.492586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7b3393-99c6-4057-8c17-664172f5eae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:41.493440 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b3393-99c6-4057-8c17-664172f5eae1 None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:41.493741 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b3393-99c6-4057-8c17-664172f5eae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:41.494020 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b3393-99c6-4057-8c17-664172f5eae1 None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:41.494754 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7b3393-99c6-4057-8c17-664172f5eae1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:41.496227 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7b3393-99c6-4057-8c17-664172f5eae1 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:41.509644 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 240/467] 10.222.0.36 () {60 vars in 1132 bytes} [Thu May 19 16:28:41 2022] POST /identity/v3/users => generated 246 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:28:41.509644 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff07ce0-8bd6-400d-8271-7cf6d81ee854 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:41.509644 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff07ce0-8bd6-400d-8271-7cf6d81ee854 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:41.509644 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff07ce0-8bd6-400d-8271-7cf6d81ee854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:41.528055 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 228/468] 10.222.0.36 () {60 vars in 1132 bytes} [Thu May 19 16:28:41 2022] POST /identity/v3/users => generated 248 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:28:41.529085 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7b3393-99c6-4057-8c17-664172f5eae1 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:41.549630 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-cff07ce0-8bd6-400d-8271-7cf6d81ee854 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:41.550886 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-cff07ce0-8bd6-400d-8271-7cf6d81ee854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:41.571209 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:41.663267 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7b3393-99c6-4057-8c17-664172f5eae1 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:41.692789 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:41.706739 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdbc02e-dd25-49ce-acc1-3daecec9c04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:41.707607 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdbc02e-dd25-49ce-acc1-3daecec9c04e None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:41.707899 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdbc02e-dd25-49ce-acc1-3daecec9c04e None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:41.708184 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdbc02e-dd25-49ce-acc1-3daecec9c04e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:41.727643 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 241/469] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:28:41.732319 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdbc02e-dd25-49ce-acc1-3daecec9c04e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:41.733942 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdbc02e-dd25-49ce-acc1-3daecec9c04e None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:41.756776 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff07ce0-8bd6-400d-8271-7cf6d81ee854 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:41.759208 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-5fdbc02e-dd25-49ce-acc1-3daecec9c04e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:41.759208 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:41.759208 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:41.759208 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:41.759208 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:41.759208 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:41.759208 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:41.759208 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:41.759208 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:41.759208 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:41.759208 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:41.759208 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:41.759208 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:28:41.759208 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:28:41.759208 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:28:41.760089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:28:41.760089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:41.760089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:41.760089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:41.760089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:41.760089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:41.760089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:41.760089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:41.760089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:41.760089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:41.760089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:41.760089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:41.760089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:41.760089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:41.760089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:41.760089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:41.760089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:41.761418 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:28:41.761418 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:28:41.761418 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:41.761418 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:41.761418 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 229/470] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:28:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:41.766648 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdbc02e-dd25-49ce-acc1-3daecec9c04e None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:41.882127 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff07ce0-8bd6-400d-8271-7cf6d81ee854 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:41.913560 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdbc02e-dd25-49ce-acc1-3daecec9c04e None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:41.935957 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d3e23b-240f-41ef-ad56-1d70349ff10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:41.937554 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3e23b-240f-41ef-ad56-1d70349ff10a None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:41.937850 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3e23b-240f-41ef-ad56-1d70349ff10a None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:41.938133 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3e23b-240f-41ef-ad56-1d70349ff10a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:41.953413 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d3e23b-240f-41ef-ad56-1d70349ff10a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:41.954979 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d3e23b-240f-41ef-ad56-1d70349ff10a None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:41.957124 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:41.962482 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046879ed-a9c9-4e72-bb26-571b45b2fab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:41.963158 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046879ed-a9c9-4e72-bb26-571b45b2fab0 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:41.963443 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046879ed-a9c9-4e72-bb26-571b45b2fab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:41.963718 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046879ed-a9c9-4e72-bb26-571b45b2fab0 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:41.964456 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046879ed-a9c9-4e72-bb26-571b45b2fab0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:41.965565 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046879ed-a9c9-4e72-bb26-571b45b2fab0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:41.966989 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046879ed-a9c9-4e72-bb26-571b45b2fab0 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:41.980407 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-99d3e23b-240f-41ef-ad56-1d70349ff10a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:41.980407 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:41.980407 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:41.980407 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:41.980407 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:41.980407 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:41.980407 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:41.980407 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:41.980407 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:41.980407 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:41.980407 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:41.980407 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:41.980407 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:28:41.980407 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:28:41.980407 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:28:41.981347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:28:41.981347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:41.981347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:41.981347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:41.981347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:41.981347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:41.981347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:41.981347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:41.981347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:28:41.981347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:41.981347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:41.981347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:41.981347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:41.981347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:41.981347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:41.981347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:41.981347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:41.982192 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:28:41.982192 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:28:41.982192 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:41.982192 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:28:41.982192 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 242/471] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:28:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:41.983298 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c2a79-976d-4d4d-87e5-2655203c19f0 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:41.983694 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c2a79-976d-4d4d-87e5-2655203c19f0 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:41.983985 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c2a79-976d-4d4d-87e5-2655203c19f0 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:41.985012 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 230/472] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:28:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:41.985427 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 243/473] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:41.990476 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c2a79-976d-4d4d-87e5-2655203c19f0 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:41.990772 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046879ed-a9c9-4e72-bb26-571b45b2fab0 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:42.035635 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:42.107480 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c2a79-976d-4d4d-87e5-2655203c19f0 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:42.109064 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046879ed-a9c9-4e72-bb26-571b45b2fab0 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:42.147697 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d0217-6ece-4bc9-a766-e145c1c42f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:42.148547 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff620397-13ea-4871-ab01-ec4c9391781e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:42.148701 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d0217-6ece-4bc9-a766-e145c1c42f1f None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:42.149051 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d0217-6ece-4bc9-a766-e145c1c42f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:42.149450 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d0217-6ece-4bc9-a766-e145c1c42f1f None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:42.149810 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff620397-13ea-4871-ab01-ec4c9391781e None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:42.150248 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff620397-13ea-4871-ab01-ec4c9391781e None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:42.150468 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1d0217-6ece-4bc9-a766-e145c1c42f1f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:42.150819 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff620397-13ea-4871-ab01-ec4c9391781e None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:42.151691 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff620397-13ea-4871-ab01-ec4c9391781e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:42.152138 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff620397-13ea-4871-ab01-ec4c9391781e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:42.152429 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1d0217-6ece-4bc9-a766-e145c1c42f1f None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:42.153835 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff620397-13ea-4871-ab01-ec4c9391781e None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:42.168004 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 231/474] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:28:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:42.169691 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a48aa-1286-4f0b-b96c-918dc2c133b5 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:42.170143 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a48aa-1286-4f0b-b96c-918dc2c133b5 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:42.170649 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a48aa-1286-4f0b-b96c-918dc2c133b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:42.182727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 244/475] 10.222.0.36 () {60 vars in 1132 bytes} [Thu May 19 16:28:41 2022] POST /identity/v3/users => generated 248 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:28:42.183869 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d0217-6ece-4bc9-a766-e145c1c42f1f None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:42.192797 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-d01a48aa-1286-4f0b-b96c-918dc2c133b5 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:42.194510 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-d01a48aa-1286-4f0b-b96c-918dc2c133b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:42.254940 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:42.309559 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 232/476] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:28:42.344309 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01a48aa-1286-4f0b-b96c-918dc2c133b5 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:42.346269 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d0217-6ece-4bc9-a766-e145c1c42f1f None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:42.385574 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c999163d-6179-4cd6-9047-2b5e28bfc888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:42.386328 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c999163d-6179-4cd6-9047-2b5e28bfc888 None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:42.386619 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c999163d-6179-4cd6-9047-2b5e28bfc888 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:42.386899 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c999163d-6179-4cd6-9047-2b5e28bfc888 None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:42.387636 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c999163d-6179-4cd6-9047-2b5e28bfc888 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:42.389146 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c999163d-6179-4cd6-9047-2b5e28bfc888 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:42.396732 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:42.436885 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 245/477] 10.222.0.36 () {60 vars in 1132 bytes} [Thu May 19 16:28:42 2022] POST /identity/v3/users => generated 249 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:28:42.511763 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01a48aa-1286-4f0b-b96c-918dc2c133b5 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:42.556532 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c35f2f8-eff6-4f7d-a4eb-49eefdbd1322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:42.557341 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c35f2f8-eff6-4f7d-a4eb-49eefdbd1322 None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:42.557520 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c35f2f8-eff6-4f7d-a4eb-49eefdbd1322 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:42.557806 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c35f2f8-eff6-4f7d-a4eb-49eefdbd1322 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:42.558604 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c35f2f8-eff6-4f7d-a4eb-49eefdbd1322 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:42.560059 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c35f2f8-eff6-4f7d-a4eb-49eefdbd1322 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:42.568573 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.sql.core [None req-1c35f2f8-eff6-4f7d-a4eb-49eefdbd1322 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 16:28:42.568573 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 16:28:42.568573 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [parameters: {'id': '006eca34be1548f88cca200505f1a34c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 16:28:42.568573 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80467) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 16:28:42.586613 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-1c35f2f8-eff6-4f7d-a4eb-49eefdbd1322 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 16:28:42.586613 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:42.586613 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 16:28:42.586613 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 16:28:42.586613 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 16:28:42.586613 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 16:28:42.586613 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 16:28:42.586613 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application result = self._query(query) May 19 16:28:42.586613 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 16:28:42.586613 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application conn.query(q) May 19 16:28:42.586613 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 16:28:42.586613 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 16:28:42.586613 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 16:28:42.586613 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application result.read() May 19 16:28:42.586613 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 16:28:42.587569 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 16:28:42.587569 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 16:28:42.587569 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 16:28:42.587569 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 16:28:42.587569 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 16:28:42.587569 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 16:28:42.587569 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 16:28:42.587569 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 16:28:42.587569 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:42.587569 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 16:28:42.587569 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:42.587569 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:42.587569 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 16:28:42.587569 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 16:28:42.587569 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 19 16:28:42.587569 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return ref.to_dict() May 19 16:28:42.587569 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 16:28:42.587569 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application next(self.gen) May 19 16:28:42.588536 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 16:28:42.588536 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application yield resource May 19 16:28:42.588536 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 16:28:42.588536 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application next(self.gen) May 19 16:28:42.588536 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 16:28:42.588536 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self.session.rollback() May 19 16:28:42.588536 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 16:28:42.588536 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self.force_reraise() May 19 16:28:42.588536 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 16:28:42.588536 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise self.value May 19 16:28:42.588536 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 16:28:42.588536 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 16:28:42.588536 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 16:28:42.588536 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application session.commit() May 19 16:28:42.588536 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 16:28:42.588536 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 16:28:42.588536 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 16:28:42.590264 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._prepare_impl() May 19 16:28:42.590264 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 16:28:42.590264 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self.session.flush() May 19 16:28:42.590264 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 16:28:42.590264 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._flush(objects) May 19 16:28:42.590264 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 16:28:42.590264 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 16:28:42.590264 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 16:28:42.590264 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application compat.raise_( May 19 16:28:42.590264 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 16:28:42.590264 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception May 19 16:28:42.590264 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 16:28:42.590264 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application flush_context.execute() May 19 16:28:42.590264 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 16:28:42.590264 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rec.execute(self) May 19 16:28:42.590264 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 16:28:42.590264 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 16:28:42.591508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 16:28:42.591508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 16:28:42.591508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 16:28:42.591508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 16:28:42.591508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 16:28:42.591508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 16:28:42.591508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 16:28:42.591508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 16:28:42.591508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 16:28:42.591508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 16:28:42.591508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 16:28:42.591508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 16:28:42.591508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 16:28:42.591508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 16:28:42.591508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 16:28:42.591508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception May 19 16:28:42.592762 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 16:28:42.592762 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 16:28:42.592762 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 16:28:42.592762 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 16:28:42.592762 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 16:28:42.592762 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application result = self._query(query) May 19 16:28:42.592762 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 16:28:42.592762 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application conn.query(q) May 19 16:28:42.592762 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 16:28:42.592762 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 16:28:42.592762 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 16:28:42.592762 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application result.read() May 19 16:28:42.592762 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 16:28:42.592762 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 16:28:42.592762 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 16:28:42.592762 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 16:28:42.594095 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 16:28:42.594095 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 16:28:42.594095 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 16:28:42.594095 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 16:28:42.594095 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 16:28:42.594095 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 16:28:42.594095 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [parameters: {'id': '006eca34be1548f88cca200505f1a34c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 16:28:42.594095 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 16:28:42.594095 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:42.594095 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 16:28:42.594095 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:42.594095 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:42.594095 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:42.594095 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:42.594095 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:42.594095 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:42.601356 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:42.601356 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:42.601356 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:42.601356 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:42.601356 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:42.601356 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:42.601356 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 19 16:28:42.601356 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 19 16:28:42.601356 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:42.601356 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:42.601356 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 19 16:28:42.601356 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 19 16:28:42.601356 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 16:28:42.601356 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 16:28:42.601356 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 16:28:42.601356 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:42.603438 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 233/478] 10.222.0.36 () {60 vars in 1131 bytes} [Thu May 19 16:28:42 2022] POST /identity/v3/roles => generated 144 bytes in 244 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 19 16:28:42.603577 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c999163d-6179-4cd6-9047-2b5e28bfc888 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:42.724516 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c999163d-6179-4cd6-9047-2b5e28bfc888 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:42.731875 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:42.761597 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952ffef3-d0fc-4540-a711-2c8c16f993f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:42.762477 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ffef3-d0fc-4540-a711-2c8c16f993f1 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:42.762707 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ffef3-d0fc-4540-a711-2c8c16f993f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:42.762988 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ffef3-d0fc-4540-a711-2c8c16f993f1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:42.775236 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952ffef3-d0fc-4540-a711-2c8c16f993f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:28:42.784459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-952ffef3-d0fc-4540-a711-2c8c16f993f1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 16:28:42.784459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:42.784459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:42.784459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:42.784459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:42.784459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:42.784459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:42.784459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:42.784459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:42.784459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:42.784459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:42.784459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:42.784459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:28:42.784459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:28:42.784459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:28:42.784459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:28:42.785383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:28:42.785383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:28:42.785383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:28:42.785383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:28:42.785383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:42.785383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:42.785383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:42.785383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:42.785383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:42.785383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:42.785383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:42.785383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:28:42.785383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:42.785383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:42.785383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:42.785383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:42.786196 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:42.786196 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:42.786196 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:42.786196 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:42.786196 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:28:42.786196 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:28:42.786196 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:28:42.786196 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:28:42.786196 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:28:42.786196 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:28:42.786196 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 16:28:42.786196 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:28:42.786196 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 246/479] 10.222.0.36 () {56 vars in 1114 bytes} [Thu May 19 16:28:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:42.793131 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c35f2f8-eff6-4f7d-a4eb-49eefdbd1322 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:42.902573 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:42.913125 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c35f2f8-eff6-4f7d-a4eb-49eefdbd1322 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:42.952628 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafd6dc2-f71a-4ce9-ab04-c514b0e07e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:42.953336 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd6dc2-f71a-4ce9-ab04-c514b0e07e6d None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:42.953622 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd6dc2-f71a-4ce9-ab04-c514b0e07e6d None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:42.953901 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd6dc2-f71a-4ce9-ab04-c514b0e07e6d None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:42.954701 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafd6dc2-f71a-4ce9-ab04-c514b0e07e6d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:42.955018 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafd6dc2-f71a-4ce9-ab04-c514b0e07e6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:42.956459 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafd6dc2-f71a-4ce9-ab04-c514b0e07e6d None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:42.966145 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 234/480] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:28:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:28:44.236756 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab6ac4-bca2-4aec-90fe-851faa1f6d4d None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:44.237431 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab6ac4-bca2-4aec-90fe-851faa1f6d4d None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:44.237431 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab6ac4-bca2-4aec-90fe-851faa1f6d4d None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:44.238902 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 247/481] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:44.244311 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3183e82-3fbb-462e-a343-5abc4c3d3931 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:44.244844 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3183e82-3fbb-462e-a343-5abc4c3d3931 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:44.244844 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3183e82-3fbb-462e-a343-5abc4c3d3931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:44.262492 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-d3183e82-3fbb-462e-a343-5abc4c3d3931 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:44.263679 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-d3183e82-3fbb-462e-a343-5abc4c3d3931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:44.342384 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 235/482] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 16:28:44.357321 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b2f98-618c-4d40-aab6-7f1c9a279012 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:44.357592 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b2f98-618c-4d40-aab6-7f1c9a279012 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:44.357881 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b2f98-618c-4d40-aab6-7f1c9a279012 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:44.359262 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 248/483] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:44.364845 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e9f54-8c68-46ec-bfa0-4b7ef282f49e None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:44.365141 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e9f54-8c68-46ec-bfa0-4b7ef282f49e None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:44.365475 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e9f54-8c68-46ec-bfa0-4b7ef282f49e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:44.383188 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-236e9f54-8c68-46ec-bfa0-4b7ef282f49e None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:44.384377 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-236e9f54-8c68-46ec-bfa0-4b7ef282f49e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:44.402423 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b2f98-618c-4d40-aab6-7f1c9a279012 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:44.466691 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 236/484] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:28:44.507297 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236e9f54-8c68-46ec-bfa0-4b7ef282f49e None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:44.529825 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:44.624322 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236e9f54-8c68-46ec-bfa0-4b7ef282f49e None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:44.646130 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b2f98-618c-4d40-aab6-7f1c9a279012 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:44.713938 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e4b3a-1114-41d3-8313-620eb6825c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:44.714760 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e4b3a-1114-41d3-8313-620eb6825c7a None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:44.715044 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e4b3a-1114-41d3-8313-620eb6825c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:44.715320 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e4b3a-1114-41d3-8313-620eb6825c7a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:44.723619 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963163eb-4a5b-4c12-b806-24630a1014ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:44.724604 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963163eb-4a5b-4c12-b806-24630a1014ac None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:44.724896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963163eb-4a5b-4c12-b806-24630a1014ac None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:44.725204 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963163eb-4a5b-4c12-b806-24630a1014ac None admin] PATH_INFO: `/v3/roles/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:44.736363 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071e4b3a-1114-41d3-8313-620eb6825c7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:28:44.750238 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-071e4b3a-1114-41d3-8313-620eb6825c7a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 16:28:44.750238 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:44.750238 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:44.750238 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:44.750238 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:44.750238 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:44.750238 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:44.750238 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:44.750238 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:44.750238 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:44.750238 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:44.750238 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:44.750238 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:28:44.750238 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:28:44.750238 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:28:44.750238 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:28:44.751197 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:28:44.751197 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:28:44.751197 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:28:44.751197 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:28:44.751197 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:44.751197 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:44.751197 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:44.751197 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:44.751197 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:44.751197 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:44.751197 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:44.751197 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:44.751197 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:44.751197 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:44.751197 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:44.751197 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:44.752072 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:44.752072 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:44.752072 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:44.752072 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:44.752072 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:28:44.752072 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:28:44.752072 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:28:44.752072 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:28:44.752072 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:28:44.752072 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:28:44.752072 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 16:28:44.752072 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:44.752682 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 237/485] 10.222.0.36 () {56 vars in 1102 bytes} [Thu May 19 16:28:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:44.757186 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963163eb-4a5b-4c12-b806-24630a1014ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:28:44.762902 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-963163eb-4a5b-4c12-b806-24630a1014ac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 16:28:44.762902 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:44.762902 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:44.762902 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:44.762902 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:44.762902 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:44.762902 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:44.762902 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:44.762902 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:44.762902 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:44.762902 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:44.762902 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:44.762902 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:28:44.762902 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:28:44.762902 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:28:44.762902 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:28:44.763796 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:28:44.763796 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:28:44.763796 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:28:44.763796 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:28:44.763796 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:44.763796 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:44.763796 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:44.763796 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:44.763796 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:44.763796 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:44.763796 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:44.763796 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:28:44.763796 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:44.763796 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:44.763796 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:44.763796 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:44.764598 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:44.764598 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:44.764598 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:44.764598 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:44.764598 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:28:44.764598 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:28:44.764598 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:28:44.764598 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:28:44.764598 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:28:44.764598 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:28:44.764598 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 16:28:44.764598 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:28:44.765232 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e4b3a-1114-41d3-8313-620eb6825c7a None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:44.780127 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 249/486] 10.222.0.36 () {56 vars in 1102 bytes} [Thu May 19 16:28:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:44.781111 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963163eb-4a5b-4c12-b806-24630a1014ac None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:44.837633 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:44.924429 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:44.942970 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963163eb-4a5b-4c12-b806-24630a1014ac None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:44.988704 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e4b3a-1114-41d3-8313-620eb6825c7a None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:45.000541 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd588b34-c413-4114-b7ee-605b42f3dd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:45.001691 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd588b34-c413-4114-b7ee-605b42f3dd2c None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:45.002014 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd588b34-c413-4114-b7ee-605b42f3dd2c None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:45.002276 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd588b34-c413-4114-b7ee-605b42f3dd2c None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:45.003055 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd588b34-c413-4114-b7ee-605b42f3dd2c None admin] RBAC: Adding query filter params (name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:45.003372 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd588b34-c413-4114-b7ee-605b42f3dd2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:45.004907 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd588b34-c413-4114-b7ee-605b42f3dd2c None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:45.027946 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 250/487] 10.222.0.36 () {56 vars in 1103 bytes} [Thu May 19 16:28:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:45.028805 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37bf343-8634-428c-b349-701beb581ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:45.029808 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37bf343-8634-428c-b349-701beb581ade None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:45.030310 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37bf343-8634-428c-b349-701beb581ade None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:45.030778 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37bf343-8634-428c-b349-701beb581ade None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:45.031735 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37bf343-8634-428c-b349-701beb581ade None admin] RBAC: Adding query filter params (name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:45.032258 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37bf343-8634-428c-b349-701beb581ade None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:45.033567 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd588b34-c413-4114-b7ee-605b42f3dd2c None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:45.034750 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37bf343-8634-428c-b349-701beb581ade None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:45.045456 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 238/488] 10.222.0.36 () {56 vars in 1103 bytes} [Thu May 19 16:28:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:45.056200 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37bf343-8634-428c-b349-701beb581ade None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:45.168008 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:45.182974 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd588b34-c413-4114-b7ee-605b42f3dd2c None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:45.227393 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1444604d-d990-413a-bea1-61d8cf897815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:45.228551 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1444604d-d990-413a-bea1-61d8cf897815 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:45.229037 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1444604d-d990-413a-bea1-61d8cf897815 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:45.229559 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1444604d-d990-413a-bea1-61d8cf897815 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:45.243821 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1444604d-d990-413a-bea1-61d8cf897815 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:45.246119 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1444604d-d990-413a-bea1-61d8cf897815 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:45.260559 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37bf343-8634-428c-b349-701beb581ade None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:45.278067 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-1444604d-d990-413a-bea1-61d8cf897815 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:45.278067 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:45.278067 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:45.278067 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:45.278067 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:45.278067 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:45.278067 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:45.278067 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:45.278067 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:45.278067 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:45.278067 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:45.278067 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:45.278067 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:28:45.278067 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:28:45.278067 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:28:45.278954 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:28:45.278954 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:45.278954 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:45.278954 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:45.278954 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:45.278954 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:45.278954 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:45.278954 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:45.278954 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:28:45.278954 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:45.278954 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:45.278954 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:45.278954 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:45.278954 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:45.278954 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:45.278954 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:45.278954 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:45.279790 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:28:45.279790 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:28:45.279790 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:45.279790 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:28:45.285683 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 251/489] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:28:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:45.290772 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1444604d-d990-413a-bea1-61d8cf897815 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:45.307170 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e111a0-b121-4297-936e-ccbe6ac00737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:45.308635 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e111a0-b121-4297-936e-ccbe6ac00737 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:45.308923 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e111a0-b121-4297-936e-ccbe6ac00737 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:45.309239 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e111a0-b121-4297-936e-ccbe6ac00737 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:45.313758 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:45.335939 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e111a0-b121-4297-936e-ccbe6ac00737 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:45.337448 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e111a0-b121-4297-936e-ccbe6ac00737 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:45.359061 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-e1e111a0-b121-4297-936e-ccbe6ac00737 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:45.359061 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:45.359061 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:45.359061 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:45.359061 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:45.359061 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:45.359061 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:45.359061 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:45.359061 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:45.359061 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:45.359061 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:45.359061 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:45.359061 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:28:45.359061 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:28:45.359061 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:28:45.359956 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:28:45.359956 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:45.359956 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:45.359956 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:45.359956 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:45.359956 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:45.359956 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:45.359956 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:45.359956 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:45.359956 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:45.359956 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:45.359956 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:45.359956 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:45.359956 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:45.359956 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:45.359956 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:45.359956 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:45.361957 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:28:45.361957 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:28:45.361957 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:45.361957 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:45.364053 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 239/490] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:28:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:45.370659 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c3fff-26af-4543-ab52-b83d798f8161 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:45.371315 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c3fff-26af-4543-ab52-b83d798f8161 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:45.371798 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c3fff-26af-4543-ab52-b83d798f8161 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:45.373897 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 240/491] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:45.375137 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0c3fff-26af-4543-ab52-b83d798f8161 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:45.424263 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1444604d-d990-413a-bea1-61d8cf897815 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:45.461851 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3f07c3-b6df-4163-8c4d-964985060955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:45.462949 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f07c3-b6df-4163-8c4d-964985060955 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:45.463438 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f07c3-b6df-4163-8c4d-964985060955 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:45.463915 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f07c3-b6df-4163-8c4d-964985060955 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:45.464901 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3f07c3-b6df-4163-8c4d-964985060955 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:45.465476 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3f07c3-b6df-4163-8c4d-964985060955 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:45.467135 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3f07c3-b6df-4163-8c4d-964985060955 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:45.478872 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:45.483358 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 252/492] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:28:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:28:45.485100 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334525fc-1d1e-48c1-a710-d423a4c84241 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:45.486226 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334525fc-1d1e-48c1-a710-d423a4c84241 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:45.486747 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334525fc-1d1e-48c1-a710-d423a4c84241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:45.505582 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-334525fc-1d1e-48c1-a710-d423a4c84241 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:45.507157 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-334525fc-1d1e-48c1-a710-d423a4c84241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:45.522151 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0c3fff-26af-4543-ab52-b83d798f8161 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:45.567092 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64fcc35-29ae-44b1-9ff5-cef69cda6370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:45.567679 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64fcc35-29ae-44b1-9ff5-cef69cda6370 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:45.567965 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64fcc35-29ae-44b1-9ff5-cef69cda6370 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:45.568609 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64fcc35-29ae-44b1-9ff5-cef69cda6370 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:45.568969 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64fcc35-29ae-44b1-9ff5-cef69cda6370 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:45.569303 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64fcc35-29ae-44b1-9ff5-cef69cda6370 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:45.570679 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64fcc35-29ae-44b1-9ff5-cef69cda6370 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:45.588620 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 241/493] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:28:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:45.588620 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64fcc35-29ae-44b1-9ff5-cef69cda6370 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:45.588864 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 253/494] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:28:45.588864 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f71baac-857c-4843-a4cb-7dae7afc341c None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:45.588864 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f71baac-857c-4843-a4cb-7dae7afc341c None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:45.588864 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f71baac-857c-4843-a4cb-7dae7afc341c None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:45.590771 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 254/495] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:45.590771 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f71baac-857c-4843-a4cb-7dae7afc341c None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:45.648036 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:45.714223 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f71baac-857c-4843-a4cb-7dae7afc341c None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:45.716792 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64fcc35-29ae-44b1-9ff5-cef69cda6370 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:45.743947 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:45.752929 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfdf1fb-8c46-4672-baa5-071e1b8c0daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:45.754341 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfdf1fb-8c46-4672-baa5-071e1b8c0daa None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:45.754864 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfdf1fb-8c46-4672-baa5-071e1b8c0daa None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:45.755400 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfdf1fb-8c46-4672-baa5-071e1b8c0daa None admin] PATH_INFO: `/v3/users/glance` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:45.756625 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5c3949-7200-4a9d-9580-7d3766ab50d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:45.757355 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c3949-7200-4a9d-9580-7d3766ab50d7 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:45.757639 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c3949-7200-4a9d-9580-7d3766ab50d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:45.757915 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c3949-7200-4a9d-9580-7d3766ab50d7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:45.765379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5c3949-7200-4a9d-9580-7d3766ab50d7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:45.767372 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5c3949-7200-4a9d-9580-7d3766ab50d7 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:45.770132 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfdf1fb-8c46-4672-baa5-071e1b8c0daa None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:45.772997 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfdf1fb-8c46-4672-baa5-071e1b8c0daa None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:45.784052 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-0a5c3949-7200-4a9d-9580-7d3766ab50d7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 16:28:45.784052 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:45.784052 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:45.784052 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:45.784052 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:45.784052 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:45.784052 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:45.784052 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:45.784052 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:45.784052 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:45.784052 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:45.784052 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:45.784052 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 16:28:45.784052 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 16:28:45.784052 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 16:28:45.784052 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 16:28:45.785005 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:45.785005 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:45.785005 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 16:28:45.785005 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:28:45.785005 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 16:28:45.785005 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:28:45.785005 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:45.785005 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:45.785005 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:45.785005 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:45.785005 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:45.785005 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:45.785005 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:45.785005 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:45.785005 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:45.785005 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:45.785005 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:45.785005 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:45.785958 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:45.785958 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:45.785958 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 16:28:45.785958 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 16:28:45.785958 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 16:28:45.785958 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 16:28:45.785958 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 16:28:45.785958 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 16:28:45.785958 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 16:28:45.785958 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:45.785958 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 242/496] 10.222.0.36 () {56 vars in 1093 bytes} [Thu May 19 16:28:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:45.786521 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be020ee-fe7e-4fef-8a34-329578c2eb33 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:45.787288 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be020ee-fe7e-4fef-8a34-329578c2eb33 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:45.787288 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be020ee-fe7e-4fef-8a34-329578c2eb33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:45.801508 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-ddfdf1fb-8c46-4672-baa5-071e1b8c0daa None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 16:28:45.801508 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:45.801508 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:45.801508 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:45.801508 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:45.801508 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:45.801508 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:45.801508 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:45.801508 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:45.801508 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:45.801508 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:45.801508 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:45.801508 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 16:28:45.801508 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 16:28:45.801508 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 16:28:45.801508 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 16:28:45.802469 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:45.802469 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:45.802469 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 16:28:45.802469 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:28:45.802469 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 16:28:45.802469 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:28:45.802469 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:45.802469 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:45.802469 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:45.802469 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:45.802469 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:45.802469 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:28:45.802469 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:45.802469 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:45.802469 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:45.802469 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:45.802469 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:45.802469 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:45.803383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:45.803383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:45.803383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 16:28:45.803383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 16:28:45.803383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 16:28:45.803383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 16:28:45.803383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 16:28:45.803383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 16:28:45.803383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 16:28:45.803383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:28:45.803383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 255/497] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:28:45 2022] GET /identity/v3/users/glance => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:45.803936 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfdf1fb-8c46-4672-baa5-071e1b8c0daa None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:45.804625 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-5be020ee-fe7e-4fef-8a34-329578c2eb33 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:45.805862 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-5be020ee-fe7e-4fef-8a34-329578c2eb33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:45.877528 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 243/498] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:28:45.878390 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be020ee-fe7e-4fef-8a34-329578c2eb33 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:45.913395 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfdf1fb-8c46-4672-baa5-071e1b8c0daa None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:45.929705 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:45.948622 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ec54c2-2ad7-4b6d-82f0-bee86c36c159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:45.949455 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ec54c2-2ad7-4b6d-82f0-bee86c36c159 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:45.949748 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ec54c2-2ad7-4b6d-82f0-bee86c36c159 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:45.950045 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ec54c2-2ad7-4b6d-82f0-bee86c36c159 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:45.960799 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ec54c2-2ad7-4b6d-82f0-bee86c36c159 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:28:45.968750 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-81ec54c2-2ad7-4b6d-82f0-bee86c36c159 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 16:28:45.968750 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:45.968750 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:45.968750 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:45.968750 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:45.968750 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:45.968750 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:45.968750 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:45.968750 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:45.968750 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:45.968750 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:45.968750 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:45.968750 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:28:45.968750 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:28:45.968750 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:28:45.968750 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:28:45.969735 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:28:45.969735 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:28:45.969735 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:28:45.969735 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:28:45.969735 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:45.969735 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:45.969735 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:45.969735 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:45.969735 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:45.969735 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:45.969735 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:45.969735 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:28:45.969735 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:45.969735 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:45.969735 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:45.969735 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:45.971123 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:45.971123 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:45.971123 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:45.971123 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:45.971123 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:28:45.971123 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:28:45.971123 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:28:45.971123 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:28:45.971123 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:28:45.971123 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:28:45.971123 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 16:28:45.971123 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:28:45.971123 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 256/499] 10.222.0.36 () {56 vars in 1102 bytes} [Thu May 19 16:28:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:45.971783 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ec54c2-2ad7-4b6d-82f0-bee86c36c159 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:46.000162 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be020ee-fe7e-4fef-8a34-329578c2eb33 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:46.024199 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:46.037636 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f854909-9512-40ce-954b-e4300383e5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:46.038325 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f854909-9512-40ce-954b-e4300383e5e2 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:46.038633 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f854909-9512-40ce-954b-e4300383e5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:46.038876 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f854909-9512-40ce-954b-e4300383e5e2 None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:46.039737 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f854909-9512-40ce-954b-e4300383e5e2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:46.040048 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f854909-9512-40ce-954b-e4300383e5e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:46.041533 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f854909-9512-40ce-954b-e4300383e5e2 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:46.079948 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ec54c2-2ad7-4b6d-82f0-bee86c36c159 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:46.082352 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 244/500] 10.222.0.36 () {56 vars in 1133 bytes} [Thu May 19 16:28:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:46.083720 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d0cf8-a97b-418d-9e0d-cdb1c0b35ab6 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:46.084019 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d0cf8-a97b-418d-9e0d-cdb1c0b35ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:46.084310 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d0cf8-a97b-418d-9e0d-cdb1c0b35ab6 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:46.085823 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 245/501] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:46.086370 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40d0cf8-a97b-418d-9e0d-cdb1c0b35ab6 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:46.116752 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d1da26-784e-4e6d-bd07-56d9a7c46568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:46.117581 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1da26-784e-4e6d-bd07-56d9a7c46568 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:46.117965 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1da26-784e-4e6d-bd07-56d9a7c46568 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:46.118324 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1da26-784e-4e6d-bd07-56d9a7c46568 None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:46.119342 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d1da26-784e-4e6d-bd07-56d9a7c46568 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:46.119766 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d1da26-784e-4e6d-bd07-56d9a7c46568 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:46.121553 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d1da26-784e-4e6d-bd07-56d9a7c46568 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:46.161902 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 257/502] 10.222.0.36 () {56 vars in 1137 bytes} [Thu May 19 16:28:45 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:46.164555 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d1da26-784e-4e6d-bd07-56d9a7c46568 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:46.184674 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:46.193459 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40d0cf8-a97b-418d-9e0d-cdb1c0b35ab6 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:46.229377 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db26e35c-1532-43a4-89aa-b74281f2dabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:46.230193 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db26e35c-1532-43a4-89aa-b74281f2dabb None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:46.230477 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db26e35c-1532-43a4-89aa-b74281f2dabb None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:46.230753 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db26e35c-1532-43a4-89aa-b74281f2dabb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:46.241524 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db26e35c-1532-43a4-89aa-b74281f2dabb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:28:46.249202 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-db26e35c-1532-43a4-89aa-b74281f2dabb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 16:28:46.249202 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:46.249202 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:46.249202 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:46.249202 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:46.249202 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:46.249202 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:46.249202 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:46.249202 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:46.249202 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:46.249202 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:46.249202 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:46.249202 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:28:46.249202 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:28:46.249202 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:28:46.249202 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:28:46.250139 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:28:46.250139 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:28:46.250139 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:28:46.250139 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:28:46.250139 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:46.250139 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:46.250139 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:46.250139 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:46.250139 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:46.250139 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:46.250139 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:46.250139 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:46.250139 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:46.250139 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:46.250139 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:46.250139 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:46.250928 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:46.250928 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:46.250928 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:46.250928 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:46.250928 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:28:46.250928 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:28:46.250928 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:28:46.250928 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:28:46.250928 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:28:46.250928 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:28:46.250928 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 16:28:46.250928 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:46.250928 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 246/503] 10.222.0.36 () {56 vars in 1102 bytes} [Thu May 19 16:28:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:46.252163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db26e35c-1532-43a4-89aa-b74281f2dabb None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:46.274442 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d1da26-784e-4e6d-bd07-56d9a7c46568 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:46.284426 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:46.310723 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a4623c-51b8-42bd-8437-0f3b94fed746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:46.311403 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4623c-51b8-42bd-8437-0f3b94fed746 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:46.311679 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4623c-51b8-42bd-8437-0f3b94fed746 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:46.311952 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4623c-51b8-42bd-8437-0f3b94fed746 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:46.312708 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a4623c-51b8-42bd-8437-0f3b94fed746 None admin] RBAC: Adding query filter params (name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:46.313015 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a4623c-51b8-42bd-8437-0f3b94fed746 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:46.314976 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a4623c-51b8-42bd-8437-0f3b94fed746 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:46.323131 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 258/504] 10.222.0.36 () {56 vars in 1103 bytes} [Thu May 19 16:28:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:46.324376 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1714ca67-692b-43f1-93e6-6ab2adc6a9b3 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:46.324668 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1714ca67-692b-43f1-93e6-6ab2adc6a9b3 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:46.324941 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1714ca67-692b-43f1-93e6-6ab2adc6a9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:46.342604 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-1714ca67-692b-43f1-93e6-6ab2adc6a9b3 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:46.343811 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-1714ca67-692b-43f1-93e6-6ab2adc6a9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:46.359394 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db26e35c-1532-43a4-89aa-b74281f2dabb None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:46.394780 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a714c4-eb30-4c0f-811c-8563dfb554e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:46.395712 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a714c4-eb30-4c0f-811c-8563dfb554e3 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:46.395995 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a714c4-eb30-4c0f-811c-8563dfb554e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:46.396274 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a714c4-eb30-4c0f-811c-8563dfb554e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:46.412929 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 259/505] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:28:46.413719 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1714ca67-692b-43f1-93e6-6ab2adc6a9b3 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:46.421064 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:46.466142 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a714c4-eb30-4c0f-811c-8563dfb554e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:46.467650 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a714c4-eb30-4c0f-811c-8563dfb554e3 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:46.526834 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:46.530110 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1714ca67-692b-43f1-93e6-6ab2adc6a9b3 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:46.565533 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2c1c24-0ce6-4ce6-9ea6-18779d17ab5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:46.566197 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c1c24-0ce6-4ce6-9ea6-18779d17ab5e None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:46.566480 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c1c24-0ce6-4ce6-9ea6-18779d17ab5e None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:46.566760 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c1c24-0ce6-4ce6-9ea6-18779d17ab5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:46.571741 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 247/506] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:28:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 16:28:46.571741 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a714c4-eb30-4c0f-811c-8563dfb554e3 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:46.624674 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:46.640249 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2c1c24-0ce6-4ce6-9ea6-18779d17ab5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:46.641869 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2c1c24-0ce6-4ce6-9ea6-18779d17ab5e None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:46.671256 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a714c4-eb30-4c0f-811c-8563dfb554e3 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:46.707939 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7e28bd-1f99-45b0-b029-a671c1daa247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:46.708606 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7e28bd-1f99-45b0-b029-a671c1daa247 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:46.708897 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7e28bd-1f99-45b0-b029-a671c1daa247 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:46.709190 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7e28bd-1f99-45b0-b029-a671c1daa247 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:46.709971 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7e28bd-1f99-45b0-b029-a671c1daa247 None admin] RBAC: Adding query filter params (name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:46.710278 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7e28bd-1f99-45b0-b029-a671c1daa247 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:46.711664 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7e28bd-1f99-45b0-b029-a671c1daa247 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:46.724729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 248/507] 10.222.0.36 () {56 vars in 1103 bytes} [Thu May 19 16:28:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:46.724729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7e28bd-1f99-45b0-b029-a671c1daa247 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:46.741441 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:46.745244 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 260/508] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:28:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 16:28:46.746426 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2c1c24-0ce6-4ce6-9ea6-18779d17ab5e None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:46.822497 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7e28bd-1f99-45b0-b029-a671c1daa247 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:46.839013 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:46.851027 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2c1c24-0ce6-4ce6-9ea6-18779d17ab5e None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:46.858480 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb568e1c-2adc-43c1-885f-4addbff0340f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:46.859282 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb568e1c-2adc-43c1-885f-4addbff0340f None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:46.859564 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb568e1c-2adc-43c1-885f-4addbff0340f None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:46.859837 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb568e1c-2adc-43c1-885f-4addbff0340f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:46.871473 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb568e1c-2adc-43c1-885f-4addbff0340f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:46.872939 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb568e1c-2adc-43c1-885f-4addbff0340f None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:46.885554 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c1088c-c58f-433c-be07-34a364d224d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:46.886354 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1088c-c58f-433c-be07-34a364d224d6 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:46.886647 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1088c-c58f-433c-be07-34a364d224d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:46.886927 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1088c-c58f-433c-be07-34a364d224d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:46.890206 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-cb568e1c-2adc-43c1-885f-4addbff0340f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:46.890206 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:46.890206 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:46.890206 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:46.890206 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:46.890206 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:46.890206 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:46.890206 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:46.890206 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:46.890206 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:46.890206 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:46.890206 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:46.890206 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:28:46.890206 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:28:46.890206 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:28:46.891035 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:28:46.891035 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:46.891035 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:46.891035 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:46.891035 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:46.891035 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:46.891035 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:46.891035 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:46.891035 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:46.891035 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:46.891035 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:46.891035 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:46.891035 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:46.891035 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:46.891035 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:46.891035 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:46.891035 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:46.891938 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:28:46.891938 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:28:46.891938 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:46.891938 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:46.891938 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 249/509] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:28:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:46.892703 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb568e1c-2adc-43c1-885f-4addbff0340f None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:46.904856 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c1088c-c58f-433c-be07-34a364d224d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:46.906418 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c1088c-c58f-433c-be07-34a364d224d6 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:46.926365 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-94c1088c-c58f-433c-be07-34a364d224d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:46.926365 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:46.926365 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:46.926365 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:46.926365 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:46.926365 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:46.926365 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:46.926365 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:46.926365 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:46.926365 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:46.926365 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:46.926365 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:46.926365 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:28:46.926365 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:28:46.926365 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:28:46.927276 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:28:46.927276 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:46.927276 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:46.927276 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:46.927276 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:46.927276 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:46.927276 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:46.927276 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:46.927276 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:28:46.927276 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:46.927276 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:46.927276 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:46.927276 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:46.927276 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:46.927276 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:46.927276 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:46.927276 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:46.928101 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:28:46.928101 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:28:46.928101 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:46.928101 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:28:46.928101 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 261/510] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:28:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:46.928633 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c1088c-c58f-433c-be07-34a364d224d6 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:46.986901 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:47.003436 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb568e1c-2adc-43c1-885f-4addbff0340f None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:47.035652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c1088c-c58f-433c-be07-34a364d224d6 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:47.037101 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc813e7-37a4-4623-8384-adeda58988b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:47.037916 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc813e7-37a4-4623-8384-adeda58988b7 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:47.038198 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc813e7-37a4-4623-8384-adeda58988b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:47.038471 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc813e7-37a4-4623-8384-adeda58988b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:47.050141 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc813e7-37a4-4623-8384-adeda58988b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:47.051659 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc813e7-37a4-4623-8384-adeda58988b7 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:47.069729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-5dc813e7-37a4-4623-8384-adeda58988b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:47.069729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:47.069729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:47.069729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:47.069729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:47.069729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:47.069729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:47.069729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:47.069729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:47.069729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:47.069729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:47.069729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:47.069729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:28:47.069729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:28:47.069729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:28:47.070751 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:28:47.070751 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:47.070751 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:47.070751 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:47.070751 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:47.070751 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:47.070751 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:47.070751 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:47.070751 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:47.070751 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:47.070751 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:47.070751 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:47.070751 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:47.070751 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:47.070751 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:47.070751 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:47.070751 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:47.071815 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:28:47.071815 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:28:47.071815 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:47.071815 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:47.072182 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab5006c-d85b-4754-8927-88751767fddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:47.072279 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 250/511] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:28:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:47.072347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5006c-d85b-4754-8927-88751767fddc None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:47.072347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5006c-d85b-4754-8927-88751767fddc None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:47.072462 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc813e7-37a4-4623-8384-adeda58988b7 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:47.072536 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5006c-d85b-4754-8927-88751767fddc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:47.085574 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab5006c-d85b-4754-8927-88751767fddc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:47.087062 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab5006c-d85b-4754-8927-88751767fddc None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:47.095360 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:47.105384 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-fab5006c-d85b-4754-8927-88751767fddc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:47.105384 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:47.105384 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:47.105384 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:47.105384 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:47.105384 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:47.105384 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:47.105384 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:47.105384 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:47.105384 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:47.105384 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:47.105384 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:47.105384 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:28:47.105384 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:28:47.105384 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:28:47.106400 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:28:47.106400 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:47.106400 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:47.106400 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:47.106400 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:47.106400 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:47.106400 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:47.106400 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:47.106400 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:28:47.106400 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:47.106400 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:47.106400 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:47.106400 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:47.106400 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:47.106400 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:47.106400 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:47.106400 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:47.107233 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:28:47.107233 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:28:47.107233 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:47.107233 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:28:47.107233 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 262/512] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:28:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:47.107700 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab5006c-d85b-4754-8927-88751767fddc None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:47.179347 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc813e7-37a4-4623-8384-adeda58988b7 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:47.211075 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab5006c-d85b-4754-8927-88751767fddc None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:47.213361 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c2547e-d73f-445f-9658-bddf13ea4cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:47.213515 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:47.214172 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2547e-d73f-445f-9658-bddf13ea4cc6 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:47.214505 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2547e-d73f-445f-9658-bddf13ea4cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:47.214811 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2547e-d73f-445f-9658-bddf13ea4cc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:47.226767 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c2547e-d73f-445f-9658-bddf13ea4cc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:47.228290 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c2547e-d73f-445f-9658-bddf13ea4cc6 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:47.246078 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-12c2547e-d73f-445f-9658-bddf13ea4cc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:47.246078 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:47.246078 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:47.246078 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:47.246078 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:47.246078 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:47.246078 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:47.246078 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:47.246078 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:47.246078 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:47.246078 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:47.246078 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:47.246078 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:28:47.246078 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:28:47.246078 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:28:47.247163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:28:47.247163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:47.247163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:47.247163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:47.247163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:47.247163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:47.247163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:47.247163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:47.247163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:47.247163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:47.247163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:47.247163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:47.247163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:47.247163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:47.247163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:47.247163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:47.247163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:47.248036 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:28:47.248036 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:28:47.248036 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:47.248036 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:47.248252 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f75e696-12a3-4dc1-b4fb-bfee85f389dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:47.248656 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 251/513] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:28:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:47.248727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f75e696-12a3-4dc1-b4fb-bfee85f389dc None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:47.248727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f75e696-12a3-4dc1-b4fb-bfee85f389dc None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:47.248727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f75e696-12a3-4dc1-b4fb-bfee85f389dc None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:47.248902 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c2547e-d73f-445f-9658-bddf13ea4cc6 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:47.248970 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f75e696-12a3-4dc1-b4fb-bfee85f389dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:47.249186 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f75e696-12a3-4dc1-b4fb-bfee85f389dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:47.250510 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f75e696-12a3-4dc1-b4fb-bfee85f389dc None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:47.262897 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 263/514] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:28:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:47.263725 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f75e696-12a3-4dc1-b4fb-bfee85f389dc None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:47.321441 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:47.351750 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c2547e-d73f-445f-9658-bddf13ea4cc6 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:47.364808 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f75e696-12a3-4dc1-b4fb-bfee85f389dc None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:47.384857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e784d688-2e64-45e1-9615-36dde2f53745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:47.385552 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784d688-2e64-45e1-9615-36dde2f53745 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:47.385842 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784d688-2e64-45e1-9615-36dde2f53745 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:47.386113 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784d688-2e64-45e1-9615-36dde2f53745 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:47.386831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e784d688-2e64-45e1-9615-36dde2f53745 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:47.387136 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e784d688-2e64-45e1-9615-36dde2f53745 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:47.388520 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e784d688-2e64-45e1-9615-36dde2f53745 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:47.399015 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc4712-cd18-4fe5-a3c1-9cdf5d2de4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:47.399687 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc4712-cd18-4fe5-a3c1-9cdf5d2de4f1 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:47.399976 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc4712-cd18-4fe5-a3c1-9cdf5d2de4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:47.400259 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc4712-cd18-4fe5-a3c1-9cdf5d2de4f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:47.400990 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc4712-cd18-4fe5-a3c1-9cdf5d2de4f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:47.401334 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc4712-cd18-4fe5-a3c1-9cdf5d2de4f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:47.402192 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 252/515] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:28:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:47.402739 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc4712-cd18-4fe5-a3c1-9cdf5d2de4f1 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:47.403017 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e784d688-2e64-45e1-9615-36dde2f53745 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:47.417797 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 264/516] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:28:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:47.418981 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc4712-cd18-4fe5-a3c1-9cdf5d2de4f1 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:47.446889 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:47.507974 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e784d688-2e64-45e1-9615-36dde2f53745 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:47.523537 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc4712-cd18-4fe5-a3c1-9cdf5d2de4f1 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:47.538094 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:47.543780 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e13c2e-ccd6-4381-b57a-20c0bc5135de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:47.544461 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e13c2e-ccd6-4381-b57a-20c0bc5135de None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:47.544740 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e13c2e-ccd6-4381-b57a-20c0bc5135de None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:47.545019 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e13c2e-ccd6-4381-b57a-20c0bc5135de None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:47.545780 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e13c2e-ccd6-4381-b57a-20c0bc5135de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:47.546152 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e13c2e-ccd6-4381-b57a-20c0bc5135de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:47.547471 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e13c2e-ccd6-4381-b57a-20c0bc5135de None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:47.557818 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa2fa35-a850-4417-9d02-66e6eef6424b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:47.558480 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2fa35-a850-4417-9d02-66e6eef6424b None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:47.558768 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2fa35-a850-4417-9d02-66e6eef6424b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:47.559048 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2fa35-a850-4417-9d02-66e6eef6424b None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:47.559454 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 253/517] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:28:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:47.559782 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa2fa35-a850-4417-9d02-66e6eef6424b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:47.560093 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa2fa35-a850-4417-9d02-66e6eef6424b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:47.560284 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e13c2e-ccd6-4381-b57a-20c0bc5135de None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:47.561524 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa2fa35-a850-4417-9d02-66e6eef6424b None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:47.574798 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 265/518] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:28:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:47.575631 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa2fa35-a850-4417-9d02-66e6eef6424b None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:47.665260 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:47.669269 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e13c2e-ccd6-4381-b57a-20c0bc5135de None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:47.681928 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa2fa35-a850-4417-9d02-66e6eef6424b None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:47.710006 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc304ce-9f74-420b-9de1-c76146a1b219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:47.710737 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc304ce-9f74-420b-9de1-c76146a1b219 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:47.711018 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc304ce-9f74-420b-9de1-c76146a1b219 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:47.711298 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc304ce-9f74-420b-9de1-c76146a1b219 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:47.716304 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754197d8-1a01-4e67-868a-c9ecc3415ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:47.716988 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754197d8-1a01-4e67-868a-c9ecc3415ff7 None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:47.717313 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754197d8-1a01-4e67-868a-c9ecc3415ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:47.717616 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754197d8-1a01-4e67-868a-c9ecc3415ff7 None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:47.718340 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754197d8-1a01-4e67-868a-c9ecc3415ff7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:47.719508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc304ce-9f74-420b-9de1-c76146a1b219 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:47.720373 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754197d8-1a01-4e67-868a-c9ecc3415ff7 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:47.721503 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc304ce-9f74-420b-9de1-c76146a1b219 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:47.737106 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-cfc304ce-9f74-420b-9de1-c76146a1b219 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 16:28:47.737106 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:47.737106 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:47.737106 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:47.737106 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:47.737106 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:47.737106 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:47.737106 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:47.737106 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:47.737106 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:47.737106 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:47.737106 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:47.737106 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 16:28:47.737106 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 16:28:47.737106 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 16:28:47.737106 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 16:28:47.738192 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:47.738192 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:47.738192 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 16:28:47.738192 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:28:47.738192 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 16:28:47.738192 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:28:47.738192 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:47.738192 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:47.738192 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:47.738192 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:47.738192 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:47.738192 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:47.738192 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:47.738192 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:47.738192 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:47.738192 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:47.738192 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:47.738192 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:47.739071 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:47.739071 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:47.739071 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 16:28:47.739071 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 16:28:47.739071 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 16:28:47.739071 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 16:28:47.739071 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 16:28:47.739071 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 16:28:47.739071 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 16:28:47.739071 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:47.739071 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 254/519] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:28:47 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:47.739650 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc304ce-9f74-420b-9de1-c76146a1b219 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:47.749085 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 266/520] 10.222.0.36 () {60 vars in 1132 bytes} [Thu May 19 16:28:47 2022] POST /identity/v3/users => generated 247 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:28:47.749085 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754197d8-1a01-4e67-868a-c9ecc3415ff7 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:47.811473 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:47.870605 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc304ce-9f74-420b-9de1-c76146a1b219 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:47.873720 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754197d8-1a01-4e67-868a-c9ecc3415ff7 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:47.913584 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fca8d4-2b57-40bd-9429-b92910d92e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:47.914406 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fca8d4-2b57-40bd-9429-b92910d92e2e None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:47.914692 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fca8d4-2b57-40bd-9429-b92910d92e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:47.914966 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fca8d4-2b57-40bd-9429-b92910d92e2e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:47.920040 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac250c2-bb01-4485-bc68-0fc57d39d208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:47.920936 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac250c2-bb01-4485-bc68-0fc57d39d208 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:47.924542 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac250c2-bb01-4485-bc68-0fc57d39d208 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:47.924542 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac250c2-bb01-4485-bc68-0fc57d39d208 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:47.924801 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fca8d4-2b57-40bd-9429-b92910d92e2e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:47.924801 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fca8d4-2b57-40bd-9429-b92910d92e2e None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:47.933564 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac250c2-bb01-4485-bc68-0fc57d39d208 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:47.935185 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac250c2-bb01-4485-bc68-0fc57d39d208 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:47.939609 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:47.941062 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-47fca8d4-2b57-40bd-9429-b92910d92e2e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:28:47.941062 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:47.941062 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:47.941062 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:47.941062 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:47.941062 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:47.941062 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:47.941062 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:47.941062 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:47.941062 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:47.941062 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:47.941062 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:47.941062 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 16:28:47.941062 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 16:28:47.941062 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 16:28:47.942328 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 16:28:47.942328 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:47.942328 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:47.942328 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:47.942328 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:47.942328 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:47.942328 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:47.942328 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:47.942328 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:47.942328 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:47.942328 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:47.942328 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:47.942328 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:47.942328 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:47.942328 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:47.942328 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:47.942328 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:47.944248 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 16:28:47.944248 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 16:28:47.944248 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 16:28:47.944248 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 16:28:47.944248 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 16:28:47.944248 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 16:28:47.944248 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:28:47.944248 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:47.947704 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 255/521] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:28:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:47.948030 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fca8d4-2b57-40bd-9429-b92910d92e2e None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:47.955228 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-9ac250c2-bb01-4485-bc68-0fc57d39d208 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 16:28:47.955228 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:47.955228 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:47.955228 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:47.955228 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:47.955228 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:47.955228 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:47.955228 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:47.955228 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:47.955228 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:47.955228 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:47.955228 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:47.955228 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 16:28:47.955228 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 16:28:47.955228 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 16:28:47.955228 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 16:28:47.956309 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:47.956309 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:47.956309 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 16:28:47.956309 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:28:47.956309 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 16:28:47.956309 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:28:47.956309 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:47.956309 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:47.956309 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:47.956309 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:47.956309 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:47.956309 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:28:47.956309 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:47.956309 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:47.956309 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:47.956309 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:47.956309 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:47.956309 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:47.957936 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:47.957936 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:47.957936 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 16:28:47.957936 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 16:28:47.957936 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 16:28:47.957936 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 16:28:47.957936 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 16:28:47.957936 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 16:28:47.957936 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 16:28:47.957936 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:28:47.958553 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 267/522] 10.222.0.36 () {56 vars in 1102 bytes} [Thu May 19 16:28:47 2022] GET /identity/v3/users/neutron => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:47.959506 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac250c2-bb01-4485-bc68-0fc57d39d208 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:48.073048 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fca8d4-2b57-40bd-9429-b92910d92e2e None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:48.088540 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:48.094280 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac250c2-bb01-4485-bc68-0fc57d39d208 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:48.114266 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa43f86-0aa4-4c0b-97ef-a65a8e4043f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:48.115073 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa43f86-0aa4-4c0b-97ef-a65a8e4043f3 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:48.115467 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa43f86-0aa4-4c0b-97ef-a65a8e4043f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:48.115748 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa43f86-0aa4-4c0b-97ef-a65a8e4043f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:48.123201 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa43f86-0aa4-4c0b-97ef-a65a8e4043f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:48.124673 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa43f86-0aa4-4c0b-97ef-a65a8e4043f3 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:48.134559 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2aedf3-040d-4e8d-ad40-9b938f01b02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:48.135483 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2aedf3-040d-4e8d-ad40-9b938f01b02f None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:48.136028 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2aedf3-040d-4e8d-ad40-9b938f01b02f None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:48.136582 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2aedf3-040d-4e8d-ad40-9b938f01b02f None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:48.137693 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2aedf3-040d-4e8d-ad40-9b938f01b02f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:48.138187 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2aedf3-040d-4e8d-ad40-9b938f01b02f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:48.138564 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-6aa43f86-0aa4-4c0b-97ef-a65a8e4043f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:28:48.138564 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:48.138564 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:48.138564 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:48.138564 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:48.138564 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:48.138564 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:48.138564 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:48.138564 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:48.138564 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:48.138564 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:48.138564 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:48.138564 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 16:28:48.138564 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 16:28:48.138564 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 16:28:48.139370 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 16:28:48.139370 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:48.139370 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:48.139370 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:48.139370 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:48.139370 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:48.139370 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:48.139370 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:48.139370 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:48.139370 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:48.139370 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:48.139370 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:48.139370 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:48.139370 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:48.139370 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:48.139370 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:48.139370 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:48.140197 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 16:28:48.140197 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 16:28:48.140197 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 16:28:48.140197 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 16:28:48.140197 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 16:28:48.140197 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 16:28:48.140197 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:28:48.140197 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:48.140197 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 256/523] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:28:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:48.141119 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa43f86-0aa4-4c0b-97ef-a65a8e4043f3 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:48.141598 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2aedf3-040d-4e8d-ad40-9b938f01b02f None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:48.162079 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 268/524] 10.222.0.36 () {56 vars in 1137 bytes} [Thu May 19 16:28:47 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:48.162949 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2aedf3-040d-4e8d-ad40-9b938f01b02f None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:48.225358 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:48.261095 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa43f86-0aa4-4c0b-97ef-a65a8e4043f3 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:48.271194 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2aedf3-040d-4e8d-ad40-9b938f01b02f None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:48.300305 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7a5669-b705-41a2-9bf9-3c1931b93b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:48.300951 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a5669-b705-41a2-9bf9-3c1931b93b51 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:48.301272 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a5669-b705-41a2-9bf9-3c1931b93b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:48.301672 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a5669-b705-41a2-9bf9-3c1931b93b51 None admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:48.302397 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7a5669-b705-41a2-9bf9-3c1931b93b51 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:48.302790 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7a5669-b705-41a2-9bf9-3c1931b93b51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:48.304455 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7a5669-b705-41a2-9bf9-3c1931b93b51 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:48.308241 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f7ed4-7660-4680-89f2-3638744e229b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:48.309050 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f7ed4-7660-4680-89f2-3638744e229b None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:48.309367 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f7ed4-7660-4680-89f2-3638744e229b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:48.309649 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f7ed4-7660-4680-89f2-3638744e229b None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:48.310508 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721f7ed4-7660-4680-89f2-3638744e229b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:48.310826 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721f7ed4-7660-4680-89f2-3638744e229b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:48.312310 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721f7ed4-7660-4680-89f2-3638744e229b None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:48.330009 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 257/525] 10.222.0.36 () {56 vars in 1148 bytes} [Thu May 19 16:28:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:48.330009 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7a5669-b705-41a2-9bf9-3c1931b93b51 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:48.340423 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 269/526] 10.222.0.36 () {56 vars in 1139 bytes} [Thu May 19 16:28:48 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:48.340423 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f7ed4-7660-4680-89f2-3638744e229b None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:48.369234 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:48.447800 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7a5669-b705-41a2-9bf9-3c1931b93b51 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:48.455080 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f7ed4-7660-4680-89f2-3638744e229b None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:48.471692 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:48.488964 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c11d58b-a775-410a-ad0b-7790a4dec958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:48.489932 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11d58b-a775-410a-ad0b-7790a4dec958 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:48.490468 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11d58b-a775-410a-ad0b-7790a4dec958 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:48.490824 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c02b152-7bef-4412-8bf8-cf9e028b83e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:48.491084 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11d58b-a775-410a-ad0b-7790a4dec958 None admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:48.491479 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c02b152-7bef-4412-8bf8-cf9e028b83e8 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:48.491763 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c02b152-7bef-4412-8bf8-cf9e028b83e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:48.492040 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c02b152-7bef-4412-8bf8-cf9e028b83e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:48.492498 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c11d58b-a775-410a-ad0b-7790a4dec958 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:48.492950 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c11d58b-a775-410a-ad0b-7790a4dec958 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:48.494564 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c11d58b-a775-410a-ad0b-7790a4dec958 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:48.512983 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 258/527] 10.222.0.36 () {56 vars in 1148 bytes} [Thu May 19 16:28:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:48.514124 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c11d58b-a775-410a-ad0b-7790a4dec958 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:48.569676 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c02b152-7bef-4412-8bf8-cf9e028b83e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:48.571159 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c02b152-7bef-4412-8bf8-cf9e028b83e8 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:48.622254 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:48.635154 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c11d58b-a775-410a-ad0b-7790a4dec958 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:48.673673 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdac00da-966c-4bd4-a678-39cace08e7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:48.674521 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac00da-966c-4bd4-a678-39cace08e7d7 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:48.674804 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac00da-966c-4bd4-a678-39cace08e7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:48.675083 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac00da-966c-4bd4-a678-39cace08e7d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:48.675936 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdac00da-966c-4bd4-a678-39cace08e7d7 None admin] RBAC: Adding query filter params (role.id=727026b648e1429ca3697e04800b328a, scope.project.id=631c76794a124d6da22c626e590e4352, user.id=58bdeada00b74b7b8e186e6dc51e6dfb) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:48.676250 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdac00da-966c-4bd4-a678-39cace08e7d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:48.677710 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdac00da-966c-4bd4-a678-39cace08e7d7 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:48.683649 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 259/528] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:28:48 2022] GET /identity/v3/role_assignments?user.id=58bdeada00b74b7b8e186e6dc51e6dfb&scope.project.id=631c76794a124d6da22c626e590e4352&role.id=727026b648e1429ca3697e04800b328a => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:48.684443 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdac00da-966c-4bd4-a678-39cace08e7d7 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:48.708756 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 270/529] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:28:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 16:28:48.710625 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c02b152-7bef-4412-8bf8-cf9e028b83e8 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:48.745652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:48.806685 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdac00da-966c-4bd4-a678-39cace08e7d7 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:48.846187 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd621468-deac-41e6-8b91-a9fc387af991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:48.846963 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd621468-deac-41e6-8b91-a9fc387af991 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:48.847150 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd621468-deac-41e6-8b91-a9fc387af991 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:48.847408 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd621468-deac-41e6-8b91-a9fc387af991 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:48.861593 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c02b152-7bef-4412-8bf8-cf9e028b83e8 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:48.877849 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:48.912869 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bb4881-1000-4ee9-bd96-771f58a0ee2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:48.913771 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb4881-1000-4ee9-bd96-771f58a0ee2a None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:48.914433 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb4881-1000-4ee9-bd96-771f58a0ee2a None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:48.914823 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb4881-1000-4ee9-bd96-771f58a0ee2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:48.916248 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bb4881-1000-4ee9-bd96-771f58a0ee2a None admin] RBAC: Adding query filter params (role.id=727026b648e1429ca3697e04800b328a, scope.project.id=631c76794a124d6da22c626e590e4352, user.id=57a5d7151478434f86be085bcc487f0c) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:48.916673 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bb4881-1000-4ee9-bd96-771f58a0ee2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:48.918329 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bb4881-1000-4ee9-bd96-771f58a0ee2a None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:48.929237 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 271/530] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:28:48 2022] GET /identity/v3/role_assignments?user.id=57a5d7151478434f86be085bcc487f0c&scope.project.id=631c76794a124d6da22c626e590e4352&role.id=727026b648e1429ca3697e04800b328a => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:48.930469 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bb4881-1000-4ee9-bd96-771f58a0ee2a None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:48.977756 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd621468-deac-41e6-8b91-a9fc387af991 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:48.980704 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd621468-deac-41e6-8b91-a9fc387af991 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:49.089553 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:49.112288 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bb4881-1000-4ee9-bd96-771f58a0ee2a None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:49.159275 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f15cbbb-9c40-4ab6-8403-a108e91467ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:49.160918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f15cbbb-9c40-4ab6-8403-a108e91467ac None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:49.161659 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f15cbbb-9c40-4ab6-8403-a108e91467ac None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:49.162115 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f15cbbb-9c40-4ab6-8403-a108e91467ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:49.177044 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f15cbbb-9c40-4ab6-8403-a108e91467ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:49.179415 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f15cbbb-9c40-4ab6-8403-a108e91467ac None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:49.201855 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-7f15cbbb-9c40-4ab6-8403-a108e91467ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:49.201855 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:49.201855 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:49.201855 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:49.201855 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:49.201855 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:49.201855 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:49.201855 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:49.201855 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:49.201855 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:49.201855 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:49.201855 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:49.201855 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:28:49.201855 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:28:49.201855 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:28:49.202764 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:28:49.202764 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:49.202764 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:49.202764 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:49.202764 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:49.202764 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:49.202764 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:49.202764 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:49.202764 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:28:49.202764 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:49.202764 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:49.202764 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:49.202764 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:49.202764 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:49.202764 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:49.202764 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:49.202764 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:49.203593 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:28:49.203593 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:28:49.203593 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:49.203593 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:28:49.206146 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 272/531] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:28:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:49.213520 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f15cbbb-9c40-4ab6-8403-a108e91467ac None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:49.254089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 260/532] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:28:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 16:28:49.266220 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd621468-deac-41e6-8b91-a9fc387af991 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:49.338183 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f15cbbb-9c40-4ab6-8403-a108e91467ac None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:49.380700 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:49.397171 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:49.414898 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2022807-77c7-4c93-b594-629633029d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:49.415495 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2022807-77c7-4c93-b594-629633029d33 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:49.415778 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2022807-77c7-4c93-b594-629633029d33 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:49.416051 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2022807-77c7-4c93-b594-629633029d33 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:49.416811 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2022807-77c7-4c93-b594-629633029d33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:49.417104 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2022807-77c7-4c93-b594-629633029d33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:49.418547 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2022807-77c7-4c93-b594-629633029d33 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:49.437264 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd621468-deac-41e6-8b91-a9fc387af991 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:49.443637 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 273/533] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:28:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:49.450678 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2022807-77c7-4c93-b594-629633029d33 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:49.504988 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d924db-6409-4d90-8e44-9dfbbd9814d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:49.505831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d924db-6409-4d90-8e44-9dfbbd9814d9 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:49.506111 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d924db-6409-4d90-8e44-9dfbbd9814d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:49.506386 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d924db-6409-4d90-8e44-9dfbbd9814d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:49.526954 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d924db-6409-4d90-8e44-9dfbbd9814d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:49.528441 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d924db-6409-4d90-8e44-9dfbbd9814d9 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:49.558118 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-e5d924db-6409-4d90-8e44-9dfbbd9814d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:49.558118 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:49.558118 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:49.558118 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:49.558118 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:49.558118 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:49.558118 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:49.558118 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:49.558118 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:49.558118 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:49.558118 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:49.558118 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:49.558118 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:28:49.558118 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:28:49.558118 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:28:49.559087 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:28:49.559087 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:49.559087 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:49.559087 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:49.559087 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:49.559087 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:49.559087 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:49.559087 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:49.559087 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:49.559087 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:49.559087 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:49.559087 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:49.559087 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:49.559087 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:49.559087 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:49.559087 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:49.559087 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:49.559937 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:28:49.559937 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:28:49.559937 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:49.559937 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:49.559937 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 261/534] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:28:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 16:28:49.572384 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d924db-6409-4d90-8e44-9dfbbd9814d9 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:49.675116 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2022807-77c7-4c93-b594-629633029d33 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:49.698697 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d924db-6409-4d90-8e44-9dfbbd9814d9 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:49.710718 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:49.741190 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd0f070-81e4-433b-9446-362b6d72c48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:49.742552 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0f070-81e4-433b-9446-362b6d72c48d None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:49.743049 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0f070-81e4-433b-9446-362b6d72c48d None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:49.743605 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0f070-81e4-433b-9446-362b6d72c48d None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:49.744603 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd0f070-81e4-433b-9446-362b6d72c48d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:49.745137 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd0f070-81e4-433b-9446-362b6d72c48d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:49.746894 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd0f070-81e4-433b-9446-362b6d72c48d None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:49.765654 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 262/535] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:28:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:49.767829 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd0f070-81e4-433b-9446-362b6d72c48d None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:49.789901 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a390b0-f62c-4898-8b23-b0d9fd5998d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:49.790727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a390b0-f62c-4898-8b23-b0d9fd5998d0 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:49.791017 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a390b0-f62c-4898-8b23-b0d9fd5998d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:49.791301 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a390b0-f62c-4898-8b23-b0d9fd5998d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:49.822023 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a390b0-f62c-4898-8b23-b0d9fd5998d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:49.823540 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a390b0-f62c-4898-8b23-b0d9fd5998d0 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:49.861984 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-36a390b0-f62c-4898-8b23-b0d9fd5998d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:28:49.861984 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:49.861984 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:49.861984 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:49.861984 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:49.861984 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:49.861984 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:49.861984 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:49.861984 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:49.861984 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:49.861984 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:49.861984 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:49.861984 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 16:28:49.861984 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 16:28:49.861984 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 16:28:49.862925 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 16:28:49.862925 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:49.862925 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:49.862925 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:49.862925 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:49.862925 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:49.862925 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:49.862925 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:49.862925 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:28:49.862925 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:49.862925 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:49.862925 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:49.862925 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:49.862925 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:49.862925 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:49.862925 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:49.862925 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:49.863857 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 16:28:49.863857 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 16:28:49.863857 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 16:28:49.863857 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 16:28:49.863857 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 16:28:49.863857 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 16:28:49.863857 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:28:49.863857 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:28:49.877189 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 274/536] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:28:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 423 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 16:28:49.885047 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd0f070-81e4-433b-9446-362b6d72c48d None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:49.890768 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a390b0-f62c-4898-8b23-b0d9fd5998d0 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:49.938978 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a183b6e-418a-4d2e-bf4a-d3da60d7c8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:49.940159 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a183b6e-418a-4d2e-bf4a-d3da60d7c8f5 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:49.942612 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a183b6e-418a-4d2e-bf4a-d3da60d7c8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:49.943405 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a183b6e-418a-4d2e-bf4a-d3da60d7c8f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:49.962464 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a183b6e-418a-4d2e-bf4a-d3da60d7c8f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:49.964319 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a183b6e-418a-4d2e-bf4a-d3da60d7c8f5 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:49.994730 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-7a183b6e-418a-4d2e-bf4a-d3da60d7c8f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:28:49.994730 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:49.994730 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:49.994730 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:49.994730 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:49.994730 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:49.994730 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:49.994730 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:49.994730 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:49.994730 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:49.994730 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:49.994730 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:49.994730 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 16:28:49.994730 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 16:28:49.994730 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 16:28:49.995676 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 16:28:49.995676 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:49.995676 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:49.995676 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:49.995676 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:49.995676 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:49.995676 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:49.995676 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:49.995676 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:49.995676 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:49.995676 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:49.995676 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:49.995676 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:49.995676 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:49.995676 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:49.995676 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:49.995676 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:49.996537 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 16:28:49.996537 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 16:28:49.996537 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 16:28:49.996537 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 16:28:49.996537 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 16:28:49.996537 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 16:28:49.996537 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:28:49.996537 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:50.003852 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 263/537] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:28:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:50.006706 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a183b6e-418a-4d2e-bf4a-d3da60d7c8f5 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:50.033515 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:50.053558 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:50.132982 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a183b6e-418a-4d2e-bf4a-d3da60d7c8f5 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:50.155792 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a390b0-f62c-4898-8b23-b0d9fd5998d0 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:50.172001 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de01865-aeb8-4125-90e8-07f90092007a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:50.172676 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de01865-aeb8-4125-90e8-07f90092007a None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:50.173044 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de01865-aeb8-4125-90e8-07f90092007a None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:50.173358 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de01865-aeb8-4125-90e8-07f90092007a None admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:50.174168 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de01865-aeb8-4125-90e8-07f90092007a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:50.174482 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de01865-aeb8-4125-90e8-07f90092007a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:50.176393 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de01865-aeb8-4125-90e8-07f90092007a None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:50.193446 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 264/538] 10.222.0.36 () {56 vars in 1148 bytes} [Thu May 19 16:28:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:50.193446 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299dbbc-84f7-4c7f-ad79-96c9171a2d7e None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:50.193446 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299dbbc-84f7-4c7f-ad79-96c9171a2d7e None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:50.193446 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299dbbc-84f7-4c7f-ad79-96c9171a2d7e None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:50.193446 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 265/539] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:50.198921 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4299dbbc-84f7-4c7f-ad79-96c9171a2d7e None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:50.212193 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febfd7b2-9e8d-48b0-9b4e-6b01c8bcbd54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:50.213588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febfd7b2-9e8d-48b0-9b4e-6b01c8bcbd54 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:50.214030 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febfd7b2-9e8d-48b0-9b4e-6b01c8bcbd54 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:50.214758 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febfd7b2-9e8d-48b0-9b4e-6b01c8bcbd54 None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:50.215726 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febfd7b2-9e8d-48b0-9b4e-6b01c8bcbd54 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:50.216429 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febfd7b2-9e8d-48b0-9b4e-6b01c8bcbd54 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:50.218430 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febfd7b2-9e8d-48b0-9b4e-6b01c8bcbd54 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:50.249454 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 275/540] 10.222.0.36 () {56 vars in 1148 bytes} [Thu May 19 16:28:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:50.249454 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e0b0a-f20c-49fb-b34e-39e75d8a46bc None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:50.249454 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e0b0a-f20c-49fb-b34e-39e75d8a46bc None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:50.249454 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e0b0a-f20c-49fb-b34e-39e75d8a46bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:50.267828 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-4a2e0b0a-f20c-49fb-b34e-39e75d8a46bc None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:50.269036 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-4a2e0b0a-f20c-49fb-b34e-39e75d8a46bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:50.315382 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:50.335616 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4299dbbc-84f7-4c7f-ad79-96c9171a2d7e None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:50.349477 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 276/541] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:28:50.350541 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e0b0a-f20c-49fb-b34e-39e75d8a46bc None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:50.379376 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9371b3-b62e-48bf-82b6-42a9a7bb43be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:50.380492 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9371b3-b62e-48bf-82b6-42a9a7bb43be None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:50.381091 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9371b3-b62e-48bf-82b6-42a9a7bb43be None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:50.381760 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9371b3-b62e-48bf-82b6-42a9a7bb43be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:50.382677 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9371b3-b62e-48bf-82b6-42a9a7bb43be None admin] RBAC: Adding query filter params (role.id=727026b648e1429ca3697e04800b328a, scope.project.id=631c76794a124d6da22c626e590e4352, user.id=33472643b65e4fe08fd5b3aa2c618133) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:50.383247 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9371b3-b62e-48bf-82b6-42a9a7bb43be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:50.384790 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9371b3-b62e-48bf-82b6-42a9a7bb43be None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:50.397309 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 266/542] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:28:50 2022] GET /identity/v3/role_assignments?user.id=33472643b65e4fe08fd5b3aa2c618133&scope.project.id=631c76794a124d6da22c626e590e4352&role.id=727026b648e1429ca3697e04800b328a => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:50.398929 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9371b3-b62e-48bf-82b6-42a9a7bb43be None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:50.446767 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:50.530993 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e0b0a-f20c-49fb-b34e-39e75d8a46bc None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:50.573308 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b61b51-4476-43f9-a6d5-a50f7ac2b54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:50.574038 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b61b51-4476-43f9-a6d5-a50f7ac2b54c None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:50.574330 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b61b51-4476-43f9-a6d5-a50f7ac2b54c None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:50.574965 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b61b51-4476-43f9-a6d5-a50f7ac2b54c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:50.575859 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b61b51-4476-43f9-a6d5-a50f7ac2b54c None admin] RBAC: Adding query filter params (role.id=727026b648e1429ca3697e04800b328a, scope.project.id=631c76794a124d6da22c626e590e4352, user.id=2cc96d859a344042a4251a4104419855) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:50.576202 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b61b51-4476-43f9-a6d5-a50f7ac2b54c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:50.577771 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b61b51-4476-43f9-a6d5-a50f7ac2b54c None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:50.589536 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 277/543] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:28:50 2022] GET /identity/v3/role_assignments?user.id=2cc96d859a344042a4251a4104419855&scope.project.id=631c76794a124d6da22c626e590e4352&role.id=727026b648e1429ca3697e04800b328a => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:50.684020 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9371b3-b62e-48bf-82b6-42a9a7bb43be None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:50.733582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305fa669-0d10-45d3-b9df-31ca6e442380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:50.734321 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305fa669-0d10-45d3-b9df-31ca6e442380 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:50.734608 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305fa669-0d10-45d3-b9df-31ca6e442380 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:50.734889 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305fa669-0d10-45d3-b9df-31ca6e442380 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:50.758525 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305fa669-0d10-45d3-b9df-31ca6e442380 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:28:50.766681 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-305fa669-0d10-45d3-b9df-31ca6e442380 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 16:28:50.766681 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:50.766681 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:50.766681 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:50.766681 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:50.766681 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:50.766681 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:50.766681 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:50.766681 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:50.766681 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:50.766681 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:50.766681 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:50.766681 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:28:50.766681 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:28:50.766681 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:28:50.766681 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:28:50.767716 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:28:50.767716 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:28:50.767716 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:28:50.767716 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:28:50.767716 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:50.767716 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:50.767716 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:50.767716 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:50.767716 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:50.767716 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:50.767716 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:50.767716 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:50.767716 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:50.767716 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:50.767716 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:50.767716 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:50.768580 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:50.768580 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:50.768580 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:50.768580 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:50.768580 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:28:50.768580 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:28:50.768580 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:28:50.768580 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:28:50.768580 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:28:50.768580 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:28:50.768580 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 16:28:50.768580 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:50.768580 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 267/544] 10.222.0.36 () {56 vars in 1102 bytes} [Thu May 19 16:28:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:50.776307 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b61b51-4476-43f9-a6d5-a50f7ac2b54c None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:50.940389 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b61b51-4476-43f9-a6d5-a50f7ac2b54c None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:50.949598 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:50.993449 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1de6e2-c60a-4c01-a2d4-2c5fc7b0773b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:50.994285 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1de6e2-c60a-4c01-a2d4-2c5fc7b0773b None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:50.994582 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1de6e2-c60a-4c01-a2d4-2c5fc7b0773b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:50.994850 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1de6e2-c60a-4c01-a2d4-2c5fc7b0773b None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:50.995651 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1de6e2-c60a-4c01-a2d4-2c5fc7b0773b None admin] RBAC: Adding query filter params (name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:50.995968 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1de6e2-c60a-4c01-a2d4-2c5fc7b0773b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:50.997430 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1de6e2-c60a-4c01-a2d4-2c5fc7b0773b None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:51.011384 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 278/545] 10.222.0.36 () {56 vars in 1103 bytes} [Thu May 19 16:28:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:51.018278 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305fa669-0d10-45d3-b9df-31ca6e442380 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:51.056470 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:51.167771 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8228d75-6d34-4140-9a79-f6624d528014 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:51.168684 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8228d75-6d34-4140-9a79-f6624d528014 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:51.168980 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8228d75-6d34-4140-9a79-f6624d528014 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:51.175338 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 279/546] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 16:28:51.178684 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a642e3-22d6-4820-8c5f-893270b98da6 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:51.178980 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a642e3-22d6-4820-8c5f-893270b98da6 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:51.179265 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a642e3-22d6-4820-8c5f-893270b98da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:51.194240 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305fa669-0d10-45d3-b9df-31ca6e442380 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:51.226291 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-80a642e3-22d6-4820-8c5f-893270b98da6 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:51.227545 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-80a642e3-22d6-4820-8c5f-893270b98da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:51.237900 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81f1bc2-988d-4195-90a8-e097e5c8089b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:51.238732 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f1bc2-988d-4195-90a8-e097e5c8089b None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:51.239019 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f1bc2-988d-4195-90a8-e097e5c8089b None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:51.239295 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f1bc2-988d-4195-90a8-e097e5c8089b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:51.260018 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81f1bc2-988d-4195-90a8-e097e5c8089b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:51.261461 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81f1bc2-988d-4195-90a8-e097e5c8089b None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:51.295952 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-d81f1bc2-988d-4195-90a8-e097e5c8089b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:51.295952 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:51.295952 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:51.295952 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:51.295952 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:51.295952 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:51.295952 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:51.295952 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:51.295952 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:51.295952 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:51.295952 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:51.295952 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:51.295952 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:28:51.295952 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:28:51.295952 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:28:51.296861 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:28:51.296861 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:51.296861 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:51.296861 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:51.296861 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:51.296861 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:51.296861 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:51.296861 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:51.296861 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:51.296861 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:51.296861 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:51.296861 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:51.296861 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:51.296861 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:51.296861 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:51.296861 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:51.296861 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:51.297896 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:28:51.297896 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:28:51.297896 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:51.297896 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:51.297896 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 268/547] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:28:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:51.308331 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81f1bc2-988d-4195-90a8-e097e5c8089b None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:51.326496 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 280/548] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:28:51.352485 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a642e3-22d6-4820-8c5f-893270b98da6 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:51.405558 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:51.475944 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81f1bc2-988d-4195-90a8-e097e5c8089b None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:51.524891 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:51.527902 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a642e3-22d6-4820-8c5f-893270b98da6 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:51.531373 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac89f81-30bf-4f18-b5cf-cef94ec63590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:51.532137 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac89f81-30bf-4f18-b5cf-cef94ec63590 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:51.532423 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac89f81-30bf-4f18-b5cf-cef94ec63590 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:51.532697 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac89f81-30bf-4f18-b5cf-cef94ec63590 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:51.533472 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac89f81-30bf-4f18-b5cf-cef94ec63590 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:51.533781 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac89f81-30bf-4f18-b5cf-cef94ec63590 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:51.535178 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac89f81-30bf-4f18-b5cf-cef94ec63590 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:51.558961 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 269/549] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:28:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:51.565824 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac89f81-30bf-4f18-b5cf-cef94ec63590 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:51.600351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9573a5ce-79e7-487c-b4f9-8802c492e319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:51.601591 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9573a5ce-79e7-487c-b4f9-8802c492e319 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:51.602140 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9573a5ce-79e7-487c-b4f9-8802c492e319 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:51.602748 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9573a5ce-79e7-487c-b4f9-8802c492e319 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:51.624571 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9573a5ce-79e7-487c-b4f9-8802c492e319 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:28:51.641193 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-9573a5ce-79e7-487c-b4f9-8802c492e319 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 16:28:51.641193 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:51.641193 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:51.641193 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:51.641193 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:51.641193 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:51.641193 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:51.641193 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:51.641193 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:51.641193 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:51.641193 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:51.641193 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:51.641193 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:28:51.641193 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:28:51.641193 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:28:51.641193 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:28:51.642729 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:28:51.642729 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:28:51.642729 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:28:51.642729 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:28:51.642729 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:51.642729 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:51.642729 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:51.642729 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:51.642729 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:51.642729 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:51.642729 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:51.642729 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:28:51.642729 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:51.642729 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:51.642729 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:51.642729 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:51.643724 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:51.643724 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:51.643724 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:51.643724 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:51.643724 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:28:51.643724 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:28:51.643724 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:28:51.643724 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:28:51.643724 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:28:51.643724 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:28:51.643724 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 16:28:51.643724 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:28:51.647203 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 281/550] 10.222.0.36 () {56 vars in 1102 bytes} [Thu May 19 16:28:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:51.654318 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9573a5ce-79e7-487c-b4f9-8802c492e319 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:51.685580 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac89f81-30bf-4f18-b5cf-cef94ec63590 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:51.710764 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:51.723411 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38764fae-31a5-4793-9f75-2546eab25b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:51.724218 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38764fae-31a5-4793-9f75-2546eab25b1c None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:51.724510 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38764fae-31a5-4793-9f75-2546eab25b1c None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:51.724791 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38764fae-31a5-4793-9f75-2546eab25b1c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:51.732665 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38764fae-31a5-4793-9f75-2546eab25b1c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:51.734124 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38764fae-31a5-4793-9f75-2546eab25b1c None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:51.750068 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-38764fae-31a5-4793-9f75-2546eab25b1c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 16:28:51.750068 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:51.750068 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:51.750068 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:51.750068 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:51.750068 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:51.750068 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:51.750068 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:51.750068 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:51.750068 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:51.750068 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:51.750068 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:51.750068 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 16:28:51.750068 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 16:28:51.750068 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 16:28:51.750068 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 16:28:51.751638 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:51.751638 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:51.751638 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 16:28:51.751638 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:28:51.751638 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 16:28:51.751638 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:28:51.751638 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:51.751638 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:51.751638 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:51.751638 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:51.751638 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:51.751638 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:51.751638 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:51.751638 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:51.751638 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:51.751638 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:51.751638 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:51.751638 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:51.753385 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:51.753385 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:51.753385 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 16:28:51.753385 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 16:28:51.753385 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 16:28:51.753385 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 16:28:51.753385 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 16:28:51.753385 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 16:28:51.753385 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 16:28:51.753385 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:51.753385 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 270/551] 10.222.0.36 () {56 vars in 1096 bytes} [Thu May 19 16:28:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 16:28:51.754004 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947f6a0e-6369-4938-a105-ae46e1ec12ce None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:51.754080 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947f6a0e-6369-4938-a105-ae46e1ec12ce None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:51.754467 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947f6a0e-6369-4938-a105-ae46e1ec12ce None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:51.756038 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 271/552] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:51.765178 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e303e4f-fd3d-4d08-a229-dafffb979684 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:51.765178 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e303e4f-fd3d-4d08-a229-dafffb979684 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:51.765178 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e303e4f-fd3d-4d08-a229-dafffb979684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:51.777825 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9573a5ce-79e7-487c-b4f9-8802c492e319 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:51.780401 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-4e303e4f-fd3d-4d08-a229-dafffb979684 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:51.781733 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-4e303e4f-fd3d-4d08-a229-dafffb979684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:51.816775 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece25517-8586-4014-9abc-20a7b1f0abfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:51.817550 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece25517-8586-4014-9abc-20a7b1f0abfd None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:51.817846 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece25517-8586-4014-9abc-20a7b1f0abfd None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:51.818132 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece25517-8586-4014-9abc-20a7b1f0abfd None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:51.818952 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece25517-8586-4014-9abc-20a7b1f0abfd None admin] RBAC: Adding query filter params (name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:51.819272 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece25517-8586-4014-9abc-20a7b1f0abfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:51.820894 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece25517-8586-4014-9abc-20a7b1f0abfd None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:51.831467 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 282/553] 10.222.0.36 () {56 vars in 1103 bytes} [Thu May 19 16:28:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:28:51.832363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece25517-8586-4014-9abc-20a7b1f0abfd None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:51.856410 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:51.877305 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 272/554] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:28:51.878149 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e303e4f-fd3d-4d08-a229-dafffb979684 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:51.955874 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece25517-8586-4014-9abc-20a7b1f0abfd None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:51.993187 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899adea8-d521-477a-8f27-e8c6a16b1006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:51.993888 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899adea8-d521-477a-8f27-e8c6a16b1006 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:51.994179 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899adea8-d521-477a-8f27-e8c6a16b1006 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:51.994475 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899adea8-d521-477a-8f27-e8c6a16b1006 None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:51.995332 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899adea8-d521-477a-8f27-e8c6a16b1006 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:51.995664 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899adea8-d521-477a-8f27-e8c6a16b1006 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:51.997320 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899adea8-d521-477a-8f27-e8c6a16b1006 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:52.016243 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e303e4f-fd3d-4d08-a229-dafffb979684 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:52.019959 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 283/555] 10.222.0.36 () {56 vars in 1135 bytes} [Thu May 19 16:28:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:52.021202 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899adea8-d521-477a-8f27-e8c6a16b1006 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:52.039493 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:52.070270 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4f539f-42f0-42ef-8153-f8f26e55ab7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:52.071737 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f539f-42f0-42ef-8153-f8f26e55ab7e None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:52.072229 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f539f-42f0-42ef-8153-f8f26e55ab7e None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:52.072986 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f539f-42f0-42ef-8153-f8f26e55ab7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:52.087985 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4f539f-42f0-42ef-8153-f8f26e55ab7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:52.090400 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4f539f-42f0-42ef-8153-f8f26e55ab7e None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:52.108003 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:52.117587 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-ab4f539f-42f0-42ef-8153-f8f26e55ab7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:52.117587 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:52.117587 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:52.117587 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:52.117587 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:52.117587 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:52.117587 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:52.117587 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:52.117587 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:52.117587 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:52.117587 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:52.117587 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:52.117587 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:28:52.117587 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:28:52.117587 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:28:52.118490 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:28:52.118490 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:52.118490 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:52.118490 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:52.118490 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:52.118490 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:52.118490 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:52.118490 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:52.118490 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:52.118490 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:52.118490 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:52.118490 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:52.118490 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:52.118490 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:52.118490 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:52.118490 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:52.118490 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:52.119353 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:28:52.119353 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:28:52.119353 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:52.119353 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:52.126727 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 273/556] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:28:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:52.127956 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4f539f-42f0-42ef-8153-f8f26e55ab7e None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:52.143540 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899adea8-d521-477a-8f27-e8c6a16b1006 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:52.185435 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b44ad-2bd0-4a93-bb10-3986f0dcd59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:52.186585 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b44ad-2bd0-4a93-bb10-3986f0dcd59a None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:52.187091 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b44ad-2bd0-4a93-bb10-3986f0dcd59a None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:52.187556 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b44ad-2bd0-4a93-bb10-3986f0dcd59a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:52.200130 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5b44ad-2bd0-4a93-bb10-3986f0dcd59a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:28:52.209627 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-5c5b44ad-2bd0-4a93-bb10-3986f0dcd59a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 16:28:52.209627 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:52.209627 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:52.209627 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:52.209627 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:52.209627 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:52.209627 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:52.209627 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:52.209627 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:52.209627 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:52.209627 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:52.209627 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:52.209627 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:28:52.209627 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:28:52.209627 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:28:52.209627 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:28:52.210715 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:28:52.210715 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:28:52.210715 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:28:52.210715 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:28:52.210715 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:52.210715 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:52.210715 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:52.210715 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:52.210715 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:52.210715 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:52.210715 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:52.210715 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:28:52.210715 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:52.210715 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:52.210715 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:52.210715 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:52.211975 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:52.211975 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:52.211975 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:52.211975 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:52.211975 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:28:52.211975 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:28:52.211975 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:28:52.211975 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:28:52.211975 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:28:52.211975 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:28:52.211975 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 16:28:52.211975 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:28:52.211975 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 284/557] 10.222.0.36 () {56 vars in 1102 bytes} [Thu May 19 16:28:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:52.212654 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b44ad-2bd0-4a93-bb10-3986f0dcd59a None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:52.248777 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4f539f-42f0-42ef-8153-f8f26e55ab7e None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:52.288283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d0d20a-85bc-4faa-b343-ad8ffcd3b3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:52.289176 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0d20a-85bc-4faa-b343-ad8ffcd3b3cd None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:52.289481 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0d20a-85bc-4faa-b343-ad8ffcd3b3cd None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:52.290152 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0d20a-85bc-4faa-b343-ad8ffcd3b3cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:52.334287 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:52.342591 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b44ad-2bd0-4a93-bb10-3986f0dcd59a None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:52.375104 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d0d20a-85bc-4faa-b343-ad8ffcd3b3cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:52.376811 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d0d20a-85bc-4faa-b343-ad8ffcd3b3cd None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:52.390910 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0198155-df79-4c74-a9b9-1ca6daa967f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:52.392194 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0198155-df79-4c74-a9b9-1ca6daa967f0 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:52.392749 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0198155-df79-4c74-a9b9-1ca6daa967f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:52.393037 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0198155-df79-4c74-a9b9-1ca6daa967f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:52.394318 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0198155-df79-4c74-a9b9-1ca6daa967f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:52.394646 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0198155-df79-4c74-a9b9-1ca6daa967f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:52.396956 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0198155-df79-4c74-a9b9-1ca6daa967f0 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:52.408073 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:52.417319 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 285/558] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:28:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:52.418491 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0198155-df79-4c74-a9b9-1ca6daa967f0 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:52.493968 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 274/559] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:28:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 16:28:52.494876 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d0d20a-85bc-4faa-b343-ad8ffcd3b3cd None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:52.570636 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:52.573119 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0198155-df79-4c74-a9b9-1ca6daa967f0 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:52.612117 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d0d20a-85bc-4faa-b343-ad8ffcd3b3cd None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:52.619022 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728b9e4e-6c74-4a63-8825-3c650c5c360b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:52.619706 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728b9e4e-6c74-4a63-8825-3c650c5c360b None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:52.620471 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728b9e4e-6c74-4a63-8825-3c650c5c360b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:52.620760 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728b9e4e-6c74-4a63-8825-3c650c5c360b None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:52.621583 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728b9e4e-6c74-4a63-8825-3c650c5c360b None admin] RBAC: Adding query filter params (name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:52.621970 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728b9e4e-6c74-4a63-8825-3c650c5c360b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:52.624158 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728b9e4e-6c74-4a63-8825-3c650c5c360b None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:52.635854 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 286/560] 10.222.0.36 () {56 vars in 1103 bytes} [Thu May 19 16:28:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:52.636806 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728b9e4e-6c74-4a63-8825-3c650c5c360b None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:52.654674 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb95dd68-265d-48b2-b28d-499b13e964bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:52.655576 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb95dd68-265d-48b2-b28d-499b13e964bd None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:52.655954 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb95dd68-265d-48b2-b28d-499b13e964bd None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:52.656300 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb95dd68-265d-48b2-b28d-499b13e964bd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:52.665252 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb95dd68-265d-48b2-b28d-499b13e964bd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:52.666839 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb95dd68-265d-48b2-b28d-499b13e964bd None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:52.682299 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-fb95dd68-265d-48b2-b28d-499b13e964bd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 16:28:52.682299 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:52.682299 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:52.682299 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:52.682299 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:52.682299 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:52.682299 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:52.682299 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:52.682299 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:52.682299 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:52.682299 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:52.682299 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:52.682299 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 16:28:52.682299 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 16:28:52.682299 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 16:28:52.682299 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 16:28:52.684237 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:52.684237 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:52.684237 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 16:28:52.684237 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:28:52.684237 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 16:28:52.684237 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:28:52.684237 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:52.684237 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:52.684237 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:52.684237 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:52.684237 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:52.684237 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:52.684237 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:52.684237 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:52.684237 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:52.684237 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:52.684237 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:52.684237 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:52.687011 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:52.687011 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:52.687011 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 16:28:52.687011 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 16:28:52.687011 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 16:28:52.687011 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 16:28:52.687011 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 16:28:52.687011 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 16:28:52.687011 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 16:28:52.687011 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:52.687011 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 275/561] 10.222.0.36 () {56 vars in 1093 bytes} [Thu May 19 16:28:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:52.687011 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb95dd68-265d-48b2-b28d-499b13e964bd None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:52.711117 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:52.776531 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728b9e4e-6c74-4a63-8825-3c650c5c360b None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:52.810137 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb95dd68-265d-48b2-b28d-499b13e964bd None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:52.831616 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aeed99-b1b1-4ad3-adbe-a031abe2087f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:52.832470 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aeed99-b1b1-4ad3-adbe-a031abe2087f None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:52.832845 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aeed99-b1b1-4ad3-adbe-a031abe2087f None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:52.833129 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aeed99-b1b1-4ad3-adbe-a031abe2087f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:52.846280 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:52.846483 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aeed99-b1b1-4ad3-adbe-a031abe2087f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:52.849525 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aeed99-b1b1-4ad3-adbe-a031abe2087f None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:52.856905 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6bb235-f39c-4643-a353-358f3920d897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:52.857973 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6bb235-f39c-4643-a353-358f3920d897 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:52.858437 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6bb235-f39c-4643-a353-358f3920d897 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:52.858856 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6bb235-f39c-4643-a353-358f3920d897 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:52.871854 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6bb235-f39c-4643-a353-358f3920d897 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:52.873761 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6bb235-f39c-4643-a353-358f3920d897 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:52.877309 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-d7aeed99-b1b1-4ad3-adbe-a031abe2087f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:52.877309 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:52.877309 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:52.877309 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:52.877309 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:52.877309 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:52.877309 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:52.877309 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:52.877309 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:52.877309 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:52.877309 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:52.877309 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:52.877309 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:28:52.877309 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:28:52.877309 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:28:52.878371 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:28:52.878371 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:52.878371 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:52.878371 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:52.878371 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:52.878371 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:52.878371 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:52.878371 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:52.878371 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:28:52.878371 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:52.878371 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:52.878371 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:52.878371 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:52.878371 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:52.878371 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:52.878371 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:52.878371 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:52.879366 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:28:52.879366 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:28:52.879366 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:52.879366 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:28:52.883290 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 287/562] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:28:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:52.884240 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aeed99-b1b1-4ad3-adbe-a031abe2087f None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:52.907576 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-0e6bb235-f39c-4643-a353-358f3920d897 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:52.907576 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:52.907576 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:52.907576 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:52.907576 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:52.907576 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:52.907576 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:52.907576 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:52.907576 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:52.907576 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:52.907576 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:52.907576 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:52.907576 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:28:52.907576 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:28:52.907576 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:28:52.909685 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:28:52.909685 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:52.909685 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:52.909685 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:52.909685 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:52.909685 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:52.909685 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:52.909685 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:52.909685 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:52.909685 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:52.909685 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:52.909685 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:52.909685 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:52.909685 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:52.909685 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:52.909685 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:52.909685 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:52.912186 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:28:52.912186 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:28:52.912186 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:52.912186 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:52.918345 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 276/563] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:28:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:52.918699 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6bb235-f39c-4643-a353-358f3920d897 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:53.041683 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6bb235-f39c-4643-a353-358f3920d897 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:53.046800 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aeed99-b1b1-4ad3-adbe-a031abe2087f None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:53.061647 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:53.084826 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5855824f-140d-43b0-a22d-7ed092cfc2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:53.085202 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec61acf-c17a-4383-a379-f5baf885ba72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:53.085461 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855824f-140d-43b0-a22d-7ed092cfc2f8 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:53.085733 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855824f-140d-43b0-a22d-7ed092cfc2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:53.085850 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec61acf-c17a-4383-a379-f5baf885ba72 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:53.086004 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855824f-140d-43b0-a22d-7ed092cfc2f8 None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:53.086106 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec61acf-c17a-4383-a379-f5baf885ba72 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:53.086319 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec61acf-c17a-4383-a379-f5baf885ba72 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:53.086944 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5855824f-140d-43b0-a22d-7ed092cfc2f8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:53.087058 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec61acf-c17a-4383-a379-f5baf885ba72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:53.087255 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5855824f-140d-43b0-a22d-7ed092cfc2f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:53.087427 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec61acf-c17a-4383-a379-f5baf885ba72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:53.088654 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec61acf-c17a-4383-a379-f5baf885ba72 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:53.088754 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5855824f-140d-43b0-a22d-7ed092cfc2f8 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:53.101405 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 277/564] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:28:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:53.102251 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec61acf-c17a-4383-a379-f5baf885ba72 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:53.110066 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 288/565] 10.222.0.36 () {56 vars in 1133 bytes} [Thu May 19 16:28:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:53.112358 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc758c3c-3ac4-4c9c-a949-957644717f9e None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:53.112897 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc758c3c-3ac4-4c9c-a949-957644717f9e None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:53.113410 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc758c3c-3ac4-4c9c-a949-957644717f9e None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:53.115243 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 289/566] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:53.116210 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc758c3c-3ac4-4c9c-a949-957644717f9e None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:53.166685 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:53.227735 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc758c3c-3ac4-4c9c-a949-957644717f9e None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:53.235286 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec61acf-c17a-4383-a379-f5baf885ba72 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:53.268563 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c26782-4f54-491d-8520-4200b7dc1d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:53.269411 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c26782-4f54-491d-8520-4200b7dc1d19 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:53.269700 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c26782-4f54-491d-8520-4200b7dc1d19 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:53.269977 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c26782-4f54-491d-8520-4200b7dc1d19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:53.276889 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c2d0ea-6b74-47cc-8d5c-efdcf5c85aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:53.277605 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2d0ea-6b74-47cc-8d5c-efdcf5c85aec None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:53.278009 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2d0ea-6b74-47cc-8d5c-efdcf5c85aec None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:53.278178 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2d0ea-6b74-47cc-8d5c-efdcf5c85aec None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:53.278271 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c26782-4f54-491d-8520-4200b7dc1d19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:53.278920 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c2d0ea-6b74-47cc-8d5c-efdcf5c85aec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:53.279238 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c2d0ea-6b74-47cc-8d5c-efdcf5c85aec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:53.279729 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c26782-4f54-491d-8520-4200b7dc1d19 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:53.280756 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c2d0ea-6b74-47cc-8d5c-efdcf5c85aec None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:53.298254 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 278/567] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:28:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:53.298254 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c2d0ea-6b74-47cc-8d5c-efdcf5c85aec None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:53.298534 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-57c26782-4f54-491d-8520-4200b7dc1d19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:28:53.298534 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:53.298534 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:53.298534 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:53.298534 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:53.298534 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:53.298534 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:53.298534 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:53.298534 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:53.298534 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:53.298534 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:53.298534 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:53.298534 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 16:28:53.298534 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 16:28:53.298534 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 16:28:53.299325 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 16:28:53.299325 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:53.299325 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:53.299325 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:53.299325 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:53.299325 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:53.299325 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:53.299325 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:53.299325 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:28:53.299325 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:53.299325 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:53.299325 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:53.299325 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:53.299325 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:53.299325 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:53.299325 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:53.299325 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:53.300194 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 16:28:53.300194 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 16:28:53.300194 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 16:28:53.300194 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 16:28:53.300194 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 16:28:53.300194 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 16:28:53.300194 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:28:53.300194 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:28:53.300194 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 290/568] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:28:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:53.300194 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55f9cd-b36c-445a-a9fe-eb86d9a953b9 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:53.300194 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55f9cd-b36c-445a-a9fe-eb86d9a953b9 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:53.300813 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55f9cd-b36c-445a-a9fe-eb86d9a953b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:53.316987 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-9e55f9cd-b36c-445a-a9fe-eb86d9a953b9 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:53.318217 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-9e55f9cd-b36c-445a-a9fe-eb86d9a953b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:53.344436 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:53.402643 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 291/569] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:28:53.402643 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e55f9cd-b36c-445a-a9fe-eb86d9a953b9 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:53.419958 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:53.424869 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c2d0ea-6b74-47cc-8d5c-efdcf5c85aec None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:53.467353 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5059625-a530-47f9-a58b-cdbe6456606d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:53.468165 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5059625-a530-47f9-a58b-cdbe6456606d None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:53.468455 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5059625-a530-47f9-a58b-cdbe6456606d None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:53.468736 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5059625-a530-47f9-a58b-cdbe6456606d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:53.480389 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5059625-a530-47f9-a58b-cdbe6456606d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:53.481913 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5059625-a530-47f9-a58b-cdbe6456606d None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:53.500228 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-c5059625-a530-47f9-a58b-cdbe6456606d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:53.500228 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:53.500228 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:53.500228 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:53.500228 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:53.500228 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:53.500228 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:53.500228 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:53.500228 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:53.500228 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:53.500228 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:53.500228 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:53.500228 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:28:53.500228 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:28:53.500228 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:28:53.501853 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:28:53.501853 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:53.501853 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:53.501853 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:53.501853 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:53.501853 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:53.501853 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:53.501853 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:53.501853 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:53.501853 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:53.501853 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:53.501853 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:53.501853 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:53.501853 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:53.501853 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:53.501853 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:53.501853 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:53.502744 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:28:53.502744 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:28:53.502744 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:53.502744 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:53.502744 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 279/570] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:28:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:53.503032 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5059625-a530-47f9-a58b-cdbe6456606d None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:53.529922 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e55f9cd-b36c-445a-a9fe-eb86d9a953b9 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:53.568789 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dedd98-6bbd-459b-a23c-c5d14eee9bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:53.569492 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dedd98-6bbd-459b-a23c-c5d14eee9bfa None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:53.569794 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dedd98-6bbd-459b-a23c-c5d14eee9bfa None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:53.570052 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dedd98-6bbd-459b-a23c-c5d14eee9bfa None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:53.570855 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dedd98-6bbd-459b-a23c-c5d14eee9bfa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:53.571166 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dedd98-6bbd-459b-a23c-c5d14eee9bfa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:53.572633 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dedd98-6bbd-459b-a23c-c5d14eee9bfa None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:53.585579 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 292/571] 10.222.0.36 () {56 vars in 1148 bytes} [Thu May 19 16:28:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:53.585579 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dedd98-6bbd-459b-a23c-c5d14eee9bfa None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:53.611025 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:53.616937 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5059625-a530-47f9-a58b-cdbe6456606d None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:53.655207 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ce9c4-6897-4a17-89dc-aaeffb312ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:53.656024 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ce9c4-6897-4a17-89dc-aaeffb312ded None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:53.656309 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ce9c4-6897-4a17-89dc-aaeffb312ded None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:53.656592 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ce9c4-6897-4a17-89dc-aaeffb312ded None admin] PATH_INFO: `/v3/users/glance` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:53.664667 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5ce9c4-6897-4a17-89dc-aaeffb312ded None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:53.666180 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5ce9c4-6897-4a17-89dc-aaeffb312ded None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:53.681932 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-fb5ce9c4-6897-4a17-89dc-aaeffb312ded None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 16:28:53.681932 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:53.681932 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:53.681932 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:53.681932 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:53.681932 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:53.681932 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:53.681932 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:53.681932 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:53.681932 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:53.681932 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:53.681932 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:53.681932 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 16:28:53.681932 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 16:28:53.681932 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 16:28:53.681932 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 16:28:53.682994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:53.682994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:53.682994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 16:28:53.682994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:28:53.682994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 16:28:53.682994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:28:53.682994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:53.682994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:53.682994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:53.682994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:53.682994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:53.682994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:53.682994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:53.682994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:53.682994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:53.682994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:53.682994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:53.682994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:53.683937 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:53.683937 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:53.683937 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 16:28:53.683937 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 16:28:53.683937 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 16:28:53.683937 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 16:28:53.683937 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 16:28:53.683937 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 16:28:53.683937 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 16:28:53.683937 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:53.683937 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 280/572] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:28:53 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:53.686511 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec3c134-1bd1-431a-ae86-8efb44190cc4 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:53.687047 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec3c134-1bd1-431a-ae86-8efb44190cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:53.687047 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec3c134-1bd1-431a-ae86-8efb44190cc4 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:53.688501 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 281/573] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:53.689225 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec3c134-1bd1-431a-ae86-8efb44190cc4 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:53.697044 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dedd98-6bbd-459b-a23c-c5d14eee9bfa None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:53.701134 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:53.731181 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1fd823-b8d8-4d62-ae25-6dca192a3bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:53.731982 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1fd823-b8d8-4d62-ae25-6dca192a3bd5 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:53.732270 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1fd823-b8d8-4d62-ae25-6dca192a3bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:53.732551 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1fd823-b8d8-4d62-ae25-6dca192a3bd5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:53.743363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1fd823-b8d8-4d62-ae25-6dca192a3bd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:28:53.751021 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-fc1fd823-b8d8-4d62-ae25-6dca192a3bd5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 16:28:53.751021 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:53.751021 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:53.751021 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:53.751021 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:53.751021 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:53.751021 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:53.751021 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:53.751021 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:53.751021 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:53.751021 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:53.751021 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:53.751021 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:28:53.751021 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:28:53.751021 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:28:53.751021 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:28:53.752016 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:28:53.752016 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:28:53.752016 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:28:53.752016 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:28:53.752016 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:53.752016 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:53.752016 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:53.752016 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:53.752016 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:53.752016 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:53.752016 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:53.752016 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:28:53.752016 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:53.752016 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:53.752016 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:53.752016 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:53.752850 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:53.752850 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:53.752850 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:53.752850 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:53.752850 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:28:53.752850 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:28:53.752850 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:28:53.752850 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:28:53.752850 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:28:53.752850 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:28:53.752850 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 16:28:53.752850 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:28:53.752850 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 293/574] 10.222.0.36 () {56 vars in 1102 bytes} [Thu May 19 16:28:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:53.753584 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1fd823-b8d8-4d62-ae25-6dca192a3bd5 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:53.795442 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec3c134-1bd1-431a-ae86-8efb44190cc4 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:53.831132 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dde24d-3086-4269-af34-502c1ee849e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:53.832106 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dde24d-3086-4269-af34-502c1ee849e7 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:53.832392 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dde24d-3086-4269-af34-502c1ee849e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:53.832666 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dde24d-3086-4269-af34-502c1ee849e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:53.833423 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dde24d-3086-4269-af34-502c1ee849e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:53.833730 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dde24d-3086-4269-af34-502c1ee849e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:53.834314 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:53.835098 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dde24d-3086-4269-af34-502c1ee849e7 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:53.846771 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 282/575] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:28:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:53.847589 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dde24d-3086-4269-af34-502c1ee849e7 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:53.858012 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1fd823-b8d8-4d62-ae25-6dca192a3bd5 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:53.893805 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab8869e-cb4a-47ec-8149-d959b25a6310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:53.894483 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8869e-cb4a-47ec-8149-d959b25a6310 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:53.894763 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8869e-cb4a-47ec-8149-d959b25a6310 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:53.895041 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8869e-cb4a-47ec-8149-d959b25a6310 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:53.896150 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab8869e-cb4a-47ec-8149-d959b25a6310 None admin] RBAC: Adding query filter params (role.id=727026b648e1429ca3697e04800b328a, scope.project.id=631c76794a124d6da22c626e590e4352, user.id=ffe7cebc6f3b40d8a8e1866ed2da9822) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:53.896464 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab8869e-cb4a-47ec-8149-d959b25a6310 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:53.897954 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab8869e-cb4a-47ec-8149-d959b25a6310 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:53.904291 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 294/576] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:28:53 2022] GET /identity/v3/role_assignments?user.id=ffe7cebc6f3b40d8a8e1866ed2da9822&scope.project.id=631c76794a124d6da22c626e590e4352&role.id=727026b648e1429ca3697e04800b328a => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:53.905599 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce98c46-27e0-4a4c-8de8-de178366ce20 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:53.905892 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce98c46-27e0-4a4c-8de8-de178366ce20 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:53.906173 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce98c46-27e0-4a4c-8de8-de178366ce20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:53.924730 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-dce98c46-27e0-4a4c-8de8-de178366ce20 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:53.926059 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-dce98c46-27e0-4a4c-8de8-de178366ce20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:53.937645 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:53.977507 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dde24d-3086-4269-af34-502c1ee849e7 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:54.010143 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 295/577] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:28:54.011111 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce98c46-27e0-4a4c-8de8-de178366ce20 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:54.034766 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c91e371-516b-4c5d-98e5-9356d34bc968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:54.035472 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91e371-516b-4c5d-98e5-9356d34bc968 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:54.035762 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91e371-516b-4c5d-98e5-9356d34bc968 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:54.036041 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91e371-516b-4c5d-98e5-9356d34bc968 None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:54.036918 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c91e371-516b-4c5d-98e5-9356d34bc968 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:54.037265 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c91e371-516b-4c5d-98e5-9356d34bc968 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:54.041202 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c91e371-516b-4c5d-98e5-9356d34bc968 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:54.060941 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 283/578] 10.222.0.36 () {56 vars in 1137 bytes} [Thu May 19 16:28:53 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:54.066058 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c91e371-516b-4c5d-98e5-9356d34bc968 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:54.103765 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:54.132520 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce98c46-27e0-4a4c-8de8-de178366ce20 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:54.170701 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b455f1-458f-41e4-a0ea-df6f4ba4473b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:54.171295 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b455f1-458f-41e4-a0ea-df6f4ba4473b None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:54.171576 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b455f1-458f-41e4-a0ea-df6f4ba4473b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:54.171849 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b455f1-458f-41e4-a0ea-df6f4ba4473b None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:54.172609 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b455f1-458f-41e4-a0ea-df6f4ba4473b None admin] RBAC: Adding query filter params (name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:54.173478 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b455f1-458f-41e4-a0ea-df6f4ba4473b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:54.174864 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b455f1-458f-41e4-a0ea-df6f4ba4473b None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:54.183812 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 296/579] 10.222.0.36 () {56 vars in 1103 bytes} [Thu May 19 16:28:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:54.184705 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b455f1-458f-41e4-a0ea-df6f4ba4473b None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:54.190585 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c91e371-516b-4c5d-98e5-9356d34bc968 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:54.218157 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:54.228216 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a12c84-101d-4034-9a58-393d9a50f4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:54.229044 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a12c84-101d-4034-9a58-393d9a50f4bd None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:54.229367 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a12c84-101d-4034-9a58-393d9a50f4bd None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:54.229645 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a12c84-101d-4034-9a58-393d9a50f4bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:54.237844 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a12c84-101d-4034-9a58-393d9a50f4bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:54.239305 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a12c84-101d-4034-9a58-393d9a50f4bd None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:54.253443 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-40a12c84-101d-4034-9a58-393d9a50f4bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:28:54.253443 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:54.253443 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:54.253443 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:54.253443 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:54.253443 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:54.253443 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:54.253443 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:54.253443 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:54.253443 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:54.253443 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:54.253443 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:54.253443 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 16:28:54.253443 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 16:28:54.253443 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 16:28:54.254572 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 16:28:54.254572 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:54.254572 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:54.254572 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:54.254572 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:54.254572 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:54.254572 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:54.254572 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:54.254572 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:54.254572 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:54.254572 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:54.254572 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:54.254572 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:54.254572 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:54.254572 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:54.254572 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:54.254572 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:54.255595 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 16:28:54.255595 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 16:28:54.255595 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 16:28:54.255595 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 16:28:54.255595 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 16:28:54.255595 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 16:28:54.255595 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:28:54.255595 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:54.255595 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 284/580] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:28:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:54.255595 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a12c84-101d-4034-9a58-393d9a50f4bd None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:54.298936 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b455f1-458f-41e4-a0ea-df6f4ba4473b None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:54.333623 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5d5b49-87f0-4797-805d-fd059ae164f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:54.334419 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d5b49-87f0-4797-805d-fd059ae164f2 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:54.334702 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d5b49-87f0-4797-805d-fd059ae164f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:54.334977 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d5b49-87f0-4797-805d-fd059ae164f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:54.337782 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:54.347993 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5d5b49-87f0-4797-805d-fd059ae164f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:28:54.355713 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-db5d5b49-87f0-4797-805d-fd059ae164f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 16:28:54.355713 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:54.355713 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:54.355713 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:54.355713 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:54.355713 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:54.355713 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:54.355713 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:54.355713 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:54.355713 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:54.355713 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:54.355713 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:54.355713 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:28:54.355713 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:28:54.355713 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:28:54.355713 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:28:54.356865 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:28:54.356865 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:28:54.356865 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:28:54.356865 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:28:54.356865 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:54.356865 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:54.356865 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:54.356865 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:54.356865 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:54.356865 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:54.356865 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:54.356865 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:28:54.356865 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:54.356865 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:54.356865 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:54.356865 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:54.357822 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:54.357822 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:54.357822 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:54.357822 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:54.357822 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:28:54.357822 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:28:54.357822 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:28:54.357822 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:28:54.357822 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:28:54.357822 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:28:54.357822 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 16:28:54.357822 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:28:54.357822 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 297/581] 10.222.0.36 () {56 vars in 1102 bytes} [Thu May 19 16:28:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:54.358539 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5d5b49-87f0-4797-805d-fd059ae164f2 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:54.369446 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a12c84-101d-4034-9a58-393d9a50f4bd None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:54.406150 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac49a2-92f0-43d9-b07b-6a0d1fed58b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:54.406956 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac49a2-92f0-43d9-b07b-6a0d1fed58b9 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:54.407784 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac49a2-92f0-43d9-b07b-6a0d1fed58b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:54.408064 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac49a2-92f0-43d9-b07b-6a0d1fed58b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:54.419134 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ac49a2-92f0-43d9-b07b-6a0d1fed58b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:54.420589 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ac49a2-92f0-43d9-b07b-6a0d1fed58b9 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:54.438520 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-93ac49a2-92f0-43d9-b07b-6a0d1fed58b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:54.438520 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:54.438520 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:54.438520 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:54.438520 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:54.438520 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:54.438520 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:54.438520 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:54.438520 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:54.438520 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:54.438520 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:54.438520 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:54.438520 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:28:54.438520 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:28:54.438520 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:28:54.439635 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:28:54.439635 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:54.439635 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:54.439635 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:54.439635 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:54.439635 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:54.439635 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:54.439635 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:54.439635 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:54.439635 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:54.439635 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:54.439635 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:54.439635 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:54.439635 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:54.439635 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:54.439635 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:54.439635 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:54.445101 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:28:54.445101 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:28:54.445101 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:54.445101 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:54.445101 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 285/582] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:28:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:54.445101 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac49a2-92f0-43d9-b07b-6a0d1fed58b9 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:54.471573 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5d5b49-87f0-4797-805d-fd059ae164f2 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:54.489662 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:54.506960 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71403d9-7e05-470b-9266-6e4f7ce10ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:54.507776 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71403d9-7e05-470b-9266-6e4f7ce10ada None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:54.508060 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71403d9-7e05-470b-9266-6e4f7ce10ada None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:54.508336 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71403d9-7e05-470b-9266-6e4f7ce10ada None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:54.519896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71403d9-7e05-470b-9266-6e4f7ce10ada None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:54.521696 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71403d9-7e05-470b-9266-6e4f7ce10ada None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:54.540926 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-d71403d9-7e05-470b-9266-6e4f7ce10ada None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:54.540926 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:54.540926 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:54.540926 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:54.540926 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:54.540926 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:54.540926 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:54.540926 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:54.540926 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:54.540926 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:54.540926 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:54.540926 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:54.540926 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:28:54.540926 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:28:54.540926 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:28:54.541896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:28:54.541896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:54.541896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:54.541896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:54.541896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:54.541896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:54.541896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:54.541896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:54.541896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:28:54.541896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:54.541896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:54.541896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:54.541896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:54.541896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:54.541896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:54.541896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:54.541896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:54.542772 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:28:54.542772 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:28:54.542772 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:54.542772 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:28:54.548804 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 298/583] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:28:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:54.548804 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71403d9-7e05-470b-9266-6e4f7ce10ada None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:54.555508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac49a2-92f0-43d9-b07b-6a0d1fed58b9 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:54.605251 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca04757-cc9d-4258-ba23-8fa89ffe2976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:54.605938 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca04757-cc9d-4258-ba23-8fa89ffe2976 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:54.606522 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca04757-cc9d-4258-ba23-8fa89ffe2976 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:54.606831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca04757-cc9d-4258-ba23-8fa89ffe2976 None admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:54.607639 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca04757-cc9d-4258-ba23-8fa89ffe2976 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:54.607959 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca04757-cc9d-4258-ba23-8fa89ffe2976 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:54.609434 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca04757-cc9d-4258-ba23-8fa89ffe2976 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:54.613180 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:54.623339 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 286/584] 10.222.0.36 () {56 vars in 1148 bytes} [Thu May 19 16:28:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:54.624184 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca04757-cc9d-4258-ba23-8fa89ffe2976 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:54.684274 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71403d9-7e05-470b-9266-6e4f7ce10ada None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:54.722682 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9905db-6b6c-4110-a6ac-48e46ef42030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:54.723277 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9905db-6b6c-4110-a6ac-48e46ef42030 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:54.723564 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9905db-6b6c-4110-a6ac-48e46ef42030 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:54.723843 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9905db-6b6c-4110-a6ac-48e46ef42030 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:54.724776 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9905db-6b6c-4110-a6ac-48e46ef42030 None admin] RBAC: Adding query filter params (name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:54.725138 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9905db-6b6c-4110-a6ac-48e46ef42030 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:54.726788 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9905db-6b6c-4110-a6ac-48e46ef42030 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:54.731762 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:54.735640 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca04757-cc9d-4258-ba23-8fa89ffe2976 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:54.741943 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 299/585] 10.222.0.36 () {56 vars in 1103 bytes} [Thu May 19 16:28:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:54.741943 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9905db-6b6c-4110-a6ac-48e46ef42030 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:54.772540 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6160c1b-562a-4ab6-ae77-26300169e357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:54.773275 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6160c1b-562a-4ab6-ae77-26300169e357 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:54.773541 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6160c1b-562a-4ab6-ae77-26300169e357 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:54.773823 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6160c1b-562a-4ab6-ae77-26300169e357 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:54.774558 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6160c1b-562a-4ab6-ae77-26300169e357 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:54.774870 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6160c1b-562a-4ab6-ae77-26300169e357 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:54.776243 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6160c1b-562a-4ab6-ae77-26300169e357 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:54.793034 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 287/586] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:28:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:54.793034 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6160c1b-562a-4ab6-ae77-26300169e357 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:54.857455 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9905db-6b6c-4110-a6ac-48e46ef42030 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:54.869853 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:54.906064 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497db28e-fdbc-488b-ac4d-36824d98d2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:54.906834 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497db28e-fdbc-488b-ac4d-36824d98d2a0 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:54.907118 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497db28e-fdbc-488b-ac4d-36824d98d2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:54.907906 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497db28e-fdbc-488b-ac4d-36824d98d2a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:54.908679 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497db28e-fdbc-488b-ac4d-36824d98d2a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:54.908992 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497db28e-fdbc-488b-ac4d-36824d98d2a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:54.910415 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497db28e-fdbc-488b-ac4d-36824d98d2a0 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:54.911606 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6160c1b-562a-4ab6-ae77-26300169e357 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:54.924366 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 300/587] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:28:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:54.925293 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497db28e-fdbc-488b-ac4d-36824d98d2a0 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:54.956386 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79590105-1315-4d03-8487-1114f1dca875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:54.957844 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79590105-1315-4d03-8487-1114f1dca875 None admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:54.958134 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79590105-1315-4d03-8487-1114f1dca875 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:54.958570 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79590105-1315-4d03-8487-1114f1dca875 None admin] PATH_INFO: `/v3/projects/631c76794a124d6da22c626e590e4352/users/58bdeada00b74b7b8e186e6dc51e6dfb/roles/727026b648e1429ca3697e04800b328a` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:54.979294 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79590105-1315-4d03-8487-1114f1dca875 None admin] RBAC: Authorizing `identity:create_grant(project_id=631c76794a124d6da22c626e590e4352, user_id=58bdeada00b74b7b8e186e6dc51e6dfb, role_id=727026b648e1429ca3697e04800b328a)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:54.980675 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79590105-1315-4d03-8487-1114f1dca875 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:55.005192 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 288/588] 10.222.0.36 () {58 vars in 1439 bytes} [Thu May 19 16:28:54 2022] PUT /identity/v3/projects/631c76794a124d6da22c626e590e4352/users/58bdeada00b74b7b8e186e6dc51e6dfb/roles/727026b648e1429ca3697e04800b328a => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:28:55.005192 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79590105-1315-4d03-8487-1114f1dca875 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:55.018502 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:55.069936 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497db28e-fdbc-488b-ac4d-36824d98d2a0 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:55.128142 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6128794-9cfc-430f-8a58-c56c8631a290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:55.128969 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6128794-9cfc-430f-8a58-c56c8631a290 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:55.129084 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79590105-1315-4d03-8487-1114f1dca875 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:55.129300 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6128794-9cfc-430f-8a58-c56c8631a290 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:55.129583 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6128794-9cfc-430f-8a58-c56c8631a290 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:55.145889 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6128794-9cfc-430f-8a58-c56c8631a290 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:55.148298 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6128794-9cfc-430f-8a58-c56c8631a290 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:55.174231 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-a6128794-9cfc-430f-8a58-c56c8631a290 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:55.174231 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:55.174231 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:55.174231 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:55.174231 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:55.174231 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:55.174231 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:55.174231 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:55.174231 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:55.174231 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:55.174231 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:55.174231 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:55.174231 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:28:55.174231 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:28:55.174231 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:28:55.175351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:28:55.175351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:55.175351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:55.175351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:55.175351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:55.175351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:55.175351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:55.175351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:55.175351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:28:55.175351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:55.175351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:55.175351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:55.175351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:55.175351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:55.175351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:55.175351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:55.175351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:55.176529 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:28:55.176529 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:28:55.176529 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:55.176529 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:28:55.176795 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21dd709-8a3e-4df0-bdf9-94261a9a5cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:55.176886 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 301/589] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:28:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:55.176957 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21dd709-8a3e-4df0-bdf9-94261a9a5cc3 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:55.176957 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21dd709-8a3e-4df0-bdf9-94261a9a5cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:55.177088 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6128794-9cfc-430f-8a58-c56c8631a290 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:55.177180 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21dd709-8a3e-4df0-bdf9-94261a9a5cc3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:55.191341 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21dd709-8a3e-4df0-bdf9-94261a9a5cc3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:55.193945 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21dd709-8a3e-4df0-bdf9-94261a9a5cc3 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:55.207850 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:55.216389 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-a21dd709-8a3e-4df0-bdf9-94261a9a5cc3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:28:55.216389 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:55.216389 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:55.216389 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:55.216389 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:55.216389 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:55.216389 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:55.216389 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:55.216389 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:55.216389 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:55.216389 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:55.216389 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:55.216389 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 16:28:55.216389 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 16:28:55.216389 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 16:28:55.217648 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 16:28:55.217648 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:55.217648 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:55.217648 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:55.217648 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:55.217648 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:55.217648 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:55.217648 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:55.217648 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:55.217648 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:55.217648 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:55.217648 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:55.217648 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:55.217648 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:55.217648 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:55.217648 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:55.217648 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:55.218761 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 16:28:55.218761 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 16:28:55.218761 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 16:28:55.218761 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 16:28:55.218761 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 16:28:55.218761 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 16:28:55.218761 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:28:55.218761 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:55.221669 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 289/590] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:28:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:55.222714 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21dd709-8a3e-4df0-bdf9-94261a9a5cc3 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:55.311325 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:55.318938 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6128794-9cfc-430f-8a58-c56c8631a290 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:55.356736 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21dd709-8a3e-4df0-bdf9-94261a9a5cc3 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:55.358283 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6936c39-c19b-47d3-b65f-49cb6d7afbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:55.359086 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6936c39-c19b-47d3-b65f-49cb6d7afbac None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:55.359378 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6936c39-c19b-47d3-b65f-49cb6d7afbac None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:55.359659 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6936c39-c19b-47d3-b65f-49cb6d7afbac None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:55.368152 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6936c39-c19b-47d3-b65f-49cb6d7afbac None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:55.369683 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6936c39-c19b-47d3-b65f-49cb6d7afbac None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:55.386217 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-a6936c39-c19b-47d3-b65f-49cb6d7afbac None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 16:28:55.386217 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:55.386217 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:55.386217 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:55.386217 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:55.386217 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:55.386217 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:55.386217 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:55.386217 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:55.386217 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:55.386217 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:55.386217 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:55.386217 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 16:28:55.386217 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 16:28:55.386217 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 16:28:55.386217 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 16:28:55.387190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:55.387190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:55.387190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 16:28:55.387190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:28:55.387190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 16:28:55.387190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:28:55.387190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:55.387190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:55.387190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:55.387190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:55.387190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:55.387190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:28:55.387190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:55.387190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:55.387190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:55.387190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:55.387190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:55.387190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:55.388110 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:55.388110 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:55.388110 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 16:28:55.388110 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 16:28:55.388110 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 16:28:55.388110 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 16:28:55.388110 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 16:28:55.388110 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 16:28:55.388110 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 16:28:55.388110 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:28:55.392085 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 302/591] 10.222.0.36 () {56 vars in 1102 bytes} [Thu May 19 16:28:55 2022] GET /identity/v3/users/neutron => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:55.393146 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6936c39-c19b-47d3-b65f-49cb6d7afbac None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:55.404193 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06046112-d4c9-4a6f-8edb-21e0d685d76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:55.405242 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06046112-d4c9-4a6f-8edb-21e0d685d76e None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:55.406172 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06046112-d4c9-4a6f-8edb-21e0d685d76e None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:55.406632 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06046112-d4c9-4a6f-8edb-21e0d685d76e None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:55.407994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06046112-d4c9-4a6f-8edb-21e0d685d76e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:55.408758 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06046112-d4c9-4a6f-8edb-21e0d685d76e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:55.410708 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06046112-d4c9-4a6f-8edb-21e0d685d76e None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:55.429779 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 290/592] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:28:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:55.430951 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06046112-d4c9-4a6f-8edb-21e0d685d76e None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:55.496096 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:55.517484 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6936c39-c19b-47d3-b65f-49cb6d7afbac None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:55.559468 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f766caa2-904c-4abf-8c7c-940d6600caaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:55.560739 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f766caa2-904c-4abf-8c7c-940d6600caaf None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:55.561241 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f766caa2-904c-4abf-8c7c-940d6600caaf None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:55.561928 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f766caa2-904c-4abf-8c7c-940d6600caaf None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:55.564197 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f766caa2-904c-4abf-8c7c-940d6600caaf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:55.564767 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f766caa2-904c-4abf-8c7c-940d6600caaf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:55.568443 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f766caa2-904c-4abf-8c7c-940d6600caaf None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:55.569532 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06046112-d4c9-4a6f-8edb-21e0d685d76e None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:55.593266 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 303/593] 10.222.0.36 () {56 vars in 1148 bytes} [Thu May 19 16:28:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:55.594966 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f766caa2-904c-4abf-8c7c-940d6600caaf None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:55.608447 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:55.616494 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba01cb6d-ea18-41fa-a534-950227599173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:55.617271 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba01cb6d-ea18-41fa-a534-950227599173 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:55.617564 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba01cb6d-ea18-41fa-a534-950227599173 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:55.617888 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba01cb6d-ea18-41fa-a534-950227599173 None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:55.618843 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba01cb6d-ea18-41fa-a534-950227599173 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:55.619158 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba01cb6d-ea18-41fa-a534-950227599173 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:55.620674 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba01cb6d-ea18-41fa-a534-950227599173 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:55.640317 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 291/594] 10.222.0.36 () {56 vars in 1139 bytes} [Thu May 19 16:28:55 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:55.641191 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba01cb6d-ea18-41fa-a534-950227599173 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:55.715613 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f766caa2-904c-4abf-8c7c-940d6600caaf None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:55.746392 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba01cb6d-ea18-41fa-a534-950227599173 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:55.754299 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e580ec64-641e-4d32-9689-a47acd943ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:55.755466 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580ec64-641e-4d32-9689-a47acd943ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:55.755953 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580ec64-641e-4d32-9689-a47acd943ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:55.756428 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580ec64-641e-4d32-9689-a47acd943ca2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:55.765441 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e580ec64-641e-4d32-9689-a47acd943ca2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:55.767281 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e580ec64-641e-4d32-9689-a47acd943ca2 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:55.783320 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c8859-ceb7-4a03-9d4e-eb44a91edd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:55.784230 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c8859-ceb7-4a03-9d4e-eb44a91edd0a None admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:55.784505 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-e580ec64-641e-4d32-9689-a47acd943ca2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 16:28:55.784505 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:55.784505 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:55.784505 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:55.784505 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:55.784505 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:55.784505 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:55.784505 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:55.784505 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:55.784505 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:55.784505 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:55.784505 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:55.784505 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 16:28:55.784505 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 16:28:55.784505 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 16:28:55.784505 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 16:28:55.785430 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:55.785430 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:55.785430 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 16:28:55.785430 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:28:55.785430 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 16:28:55.785430 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:28:55.785430 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:55.785430 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:55.785430 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:55.785430 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:55.785430 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:55.785430 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:28:55.785430 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:55.785430 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:55.785430 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:55.785430 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:55.785430 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:55.785430 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:55.786363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:55.786363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:55.786363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 16:28:55.786363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 16:28:55.786363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 16:28:55.786363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 16:28:55.786363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 16:28:55.786363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 16:28:55.786363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 16:28:55.786363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:28:55.786898 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c8859-ceb7-4a03-9d4e-eb44a91edd0a None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:55.786898 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c8859-ceb7-4a03-9d4e-eb44a91edd0a None admin] PATH_INFO: `/v3/projects/631c76794a124d6da22c626e590e4352/users/57a5d7151478434f86be085bcc487f0c/roles/727026b648e1429ca3697e04800b328a` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:55.789306 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 304/595] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:28:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:55.790368 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e580ec64-641e-4d32-9689-a47acd943ca2 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:55.793367 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:55.803094 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c8859-ceb7-4a03-9d4e-eb44a91edd0a None admin] RBAC: Authorizing `identity:create_grant(project_id=631c76794a124d6da22c626e590e4352, user_id=57a5d7151478434f86be085bcc487f0c, role_id=727026b648e1429ca3697e04800b328a)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:55.804510 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c8859-ceb7-4a03-9d4e-eb44a91edd0a None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:55.821665 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 292/596] 10.222.0.36 () {58 vars in 1439 bytes} [Thu May 19 16:28:55 2022] PUT /identity/v3/projects/631c76794a124d6da22c626e590e4352/users/57a5d7151478434f86be085bcc487f0c/roles/727026b648e1429ca3697e04800b328a => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:28:55.822486 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c8859-ceb7-4a03-9d4e-eb44a91edd0a None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:55.868286 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:55.917035 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e580ec64-641e-4d32-9689-a47acd943ca2 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:55.938125 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c8859-ceb7-4a03-9d4e-eb44a91edd0a None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:55.958387 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7638ea54-c677-4d1c-81fc-39836c56f5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:55.959542 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7638ea54-c677-4d1c-81fc-39836c56f5e4 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:55.960047 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7638ea54-c677-4d1c-81fc-39836c56f5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:55.960614 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7638ea54-c677-4d1c-81fc-39836c56f5e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:55.975340 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7638ea54-c677-4d1c-81fc-39836c56f5e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:55.976420 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f09f5-987d-4b07-bd47-099f90d63319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:55.977114 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f09f5-987d-4b07-bd47-099f90d63319 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:55.977476 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f09f5-987d-4b07-bd47-099f90d63319 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:55.977802 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f09f5-987d-4b07-bd47-099f90d63319 None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:55.978169 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7638ea54-c677-4d1c-81fc-39836c56f5e4 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:55.978666 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886f09f5-987d-4b07-bd47-099f90d63319 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:55.978975 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886f09f5-987d-4b07-bd47-099f90d63319 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:55.980470 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886f09f5-987d-4b07-bd47-099f90d63319 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:55.998817 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-7638ea54-c677-4d1c-81fc-39836c56f5e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:55.998817 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:55.998817 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:55.998817 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:55.998817 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:55.998817 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:55.998817 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:55.998817 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:55.998817 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:55.998817 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:55.998817 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:55.998817 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:55.998817 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:28:55.998817 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:28:55.998817 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:28:55.999777 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:28:55.999777 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:55.999777 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:55.999777 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:55.999777 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:55.999777 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:55.999777 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:55.999777 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:55.999777 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:28:55.999777 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:55.999777 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:55.999777 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:55.999777 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:55.999777 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:55.999777 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:55.999777 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:55.999777 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:56.000660 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:28:56.000660 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:28:56.000660 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:56.000660 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:28:56.001726 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 293/597] 10.222.0.36 () {56 vars in 1137 bytes} [Thu May 19 16:28:55 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:56.003361 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 305/598] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:28:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:56.009241 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f09f5-987d-4b07-bd47-099f90d63319 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:56.014340 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7638ea54-c677-4d1c-81fc-39836c56f5e4 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:56.140969 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:56.148541 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:56.151787 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f09f5-987d-4b07-bd47-099f90d63319 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:56.201553 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f3a42a-568c-447a-845a-e615dfbc2e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:56.203173 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3a42a-568c-447a-845a-e615dfbc2e1b None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:56.203641 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3a42a-568c-447a-845a-e615dfbc2e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:56.204139 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3a42a-568c-447a-845a-e615dfbc2e1b None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:56.205493 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f3a42a-568c-447a-845a-e615dfbc2e1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:56.205824 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f3a42a-568c-447a-845a-e615dfbc2e1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:56.207233 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f3a42a-568c-447a-845a-e615dfbc2e1b None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:56.211350 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7638ea54-c677-4d1c-81fc-39836c56f5e4 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:56.222595 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 294/599] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:28:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:56.228189 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f3a42a-568c-447a-845a-e615dfbc2e1b None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:56.253739 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972d7ea1-649c-4a68-ba14-257a069e64a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:56.254273 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d7ea1-649c-4a68-ba14-257a069e64a0 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:56.254561 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d7ea1-649c-4a68-ba14-257a069e64a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:56.254837 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d7ea1-649c-4a68-ba14-257a069e64a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:56.267398 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972d7ea1-649c-4a68-ba14-257a069e64a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:56.269310 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972d7ea1-649c-4a68-ba14-257a069e64a0 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:56.287796 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-972d7ea1-649c-4a68-ba14-257a069e64a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:56.287796 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:56.287796 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:56.287796 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:56.287796 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:56.287796 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:56.287796 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:56.287796 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:56.287796 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:56.287796 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:56.287796 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:56.287796 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:56.287796 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:28:56.287796 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:28:56.287796 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:28:56.288736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:28:56.288736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:56.288736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:56.288736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:56.288736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:56.288736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:56.288736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:56.288736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:56.288736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:28:56.288736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:56.288736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:56.288736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:56.288736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:56.288736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:56.288736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:56.288736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:56.288736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:56.289654 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:28:56.289654 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:28:56.289654 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:56.289654 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:28:56.289654 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 306/600] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:28:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:56.290661 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb626da-8bd1-497b-9de6-96becde5b396 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:56.291330 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb626da-8bd1-497b-9de6-96becde5b396 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:56.291330 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb626da-8bd1-497b-9de6-96becde5b396 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:56.292778 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 307/601] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:56.299333 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb626da-8bd1-497b-9de6-96becde5b396 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:56.346312 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f3a42a-568c-447a-845a-e615dfbc2e1b None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:56.393627 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7892c3-f25b-4e3a-af5b-af8d45408f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:56.394816 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7892c3-f25b-4e3a-af5b-af8d45408f44 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:56.395452 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7892c3-f25b-4e3a-af5b-af8d45408f44 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:56.395933 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7892c3-f25b-4e3a-af5b-af8d45408f44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:56.406622 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7892c3-f25b-4e3a-af5b-af8d45408f44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:56.409563 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7892c3-f25b-4e3a-af5b-af8d45408f44 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:56.420071 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb626da-8bd1-497b-9de6-96becde5b396 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:56.428810 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:56.430171 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-5e7892c3-f25b-4e3a-af5b-af8d45408f44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:28:56.430171 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:56.430171 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:56.430171 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:56.430171 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:56.430171 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:56.430171 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:56.430171 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:56.430171 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:56.430171 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:56.430171 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:56.430171 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:56.430171 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 16:28:56.430171 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 16:28:56.430171 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 16:28:56.431083 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 16:28:56.431083 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:56.431083 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:56.431083 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:56.431083 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:56.431083 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:56.431083 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:56.431083 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:56.431083 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:56.431083 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:56.431083 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:56.431083 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:56.431083 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:56.431083 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:56.431083 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:56.431083 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:56.431083 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:56.432013 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 16:28:56.432013 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 16:28:56.432013 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 16:28:56.432013 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 16:28:56.432013 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 16:28:56.432013 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 16:28:56.432013 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:28:56.432013 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:56.436759 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 295/602] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:28:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 16:28:56.439551 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff60ca2-8ec8-4932-af8d-44379fec5e90 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:56.439851 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff60ca2-8ec8-4932-af8d-44379fec5e90 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:56.440135 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff60ca2-8ec8-4932-af8d-44379fec5e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:56.463137 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-6ff60ca2-8ec8-4932-af8d-44379fec5e90 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:56.464614 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-6ff60ca2-8ec8-4932-af8d-44379fec5e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:56.468051 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b62da6a-f974-4701-870d-78b11aee4019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:56.469000 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62da6a-f974-4701-870d-78b11aee4019 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:56.470115 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62da6a-f974-4701-870d-78b11aee4019 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:56.470462 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62da6a-f974-4701-870d-78b11aee4019 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:56.471203 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b62da6a-f974-4701-870d-78b11aee4019 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:56.471561 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b62da6a-f974-4701-870d-78b11aee4019 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:56.473069 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b62da6a-f974-4701-870d-78b11aee4019 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:56.489847 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 308/603] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:28:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:56.491081 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b62da6a-f974-4701-870d-78b11aee4019 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:56.492061 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:56.557287 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 296/604] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:28:56.558622 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff60ca2-8ec8-4932-af8d-44379fec5e90 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:56.621223 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b62da6a-f974-4701-870d-78b11aee4019 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:56.669873 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ffdbd-bb4c-4084-91f6-96f866150409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:56.670467 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ffdbd-bb4c-4084-91f6-96f866150409 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:56.670754 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ffdbd-bb4c-4084-91f6-96f866150409 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:56.671039 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ffdbd-bb4c-4084-91f6-96f866150409 None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:56.671848 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ffdbd-bb4c-4084-91f6-96f866150409 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:56.672163 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ffdbd-bb4c-4084-91f6-96f866150409 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:56.674806 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ffdbd-bb4c-4084-91f6-96f866150409 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:56.686459 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff60ca2-8ec8-4932-af8d-44379fec5e90 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:56.694736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 309/605] 10.222.0.36 () {56 vars in 1148 bytes} [Thu May 19 16:28:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:56.695633 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ffdbd-bb4c-4084-91f6-96f866150409 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:56.729437 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:56.738585 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61898bd4-3ae6-47de-a68c-65be5065639f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:56.740831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61898bd4-3ae6-47de-a68c-65be5065639f None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:56.741398 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61898bd4-3ae6-47de-a68c-65be5065639f None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:56.741898 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61898bd4-3ae6-47de-a68c-65be5065639f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:56.751658 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61898bd4-3ae6-47de-a68c-65be5065639f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:56.754755 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61898bd4-3ae6-47de-a68c-65be5065639f None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:56.774484 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-61898bd4-3ae6-47de-a68c-65be5065639f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:28:56.774484 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:56.774484 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:56.774484 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:56.774484 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:56.774484 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:56.774484 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:56.774484 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:56.774484 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:56.774484 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:56.774484 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:56.774484 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:56.774484 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 16:28:56.774484 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 16:28:56.774484 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 16:28:56.775603 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 16:28:56.775603 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:56.775603 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:56.775603 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:56.775603 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:56.775603 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:56.775603 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:56.775603 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:56.775603 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:56.775603 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:56.775603 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:56.775603 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:56.775603 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:56.775603 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:56.775603 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:56.775603 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:56.775603 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:56.776703 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 16:28:56.776703 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 16:28:56.776703 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 16:28:56.776703 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 16:28:56.776703 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 16:28:56.776703 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 16:28:56.776703 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:28:56.776703 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:56.779691 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 297/606] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:28:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:56.780642 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61898bd4-3ae6-47de-a68c-65be5065639f None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:56.794995 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:56.818844 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ffdbd-bb4c-4084-91f6-96f866150409 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:56.859838 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466dfcb-f140-4957-9091-32388ebc3efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:56.861242 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466dfcb-f140-4957-9091-32388ebc3efe None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:56.861658 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466dfcb-f140-4957-9091-32388ebc3efe None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:56.861902 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466dfcb-f140-4957-9091-32388ebc3efe None admin] PATH_INFO: `/v3/roles/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:56.873294 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8466dfcb-f140-4957-9091-32388ebc3efe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:28:56.881331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-8466dfcb-f140-4957-9091-32388ebc3efe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 16:28:56.881331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:56.881331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:56.881331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:56.881331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:56.881331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:56.881331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:56.881331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:56.881331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:56.881331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:56.881331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:56.881331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:56.881331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:28:56.881331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:28:56.881331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:28:56.881331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:28:56.882510 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:28:56.882510 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:28:56.882510 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:28:56.882510 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:28:56.882510 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:56.882510 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:56.882510 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:56.882510 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:56.882510 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:56.882510 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:56.882510 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:56.882510 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:28:56.882510 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:56.882510 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:56.882510 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:56.882510 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:56.883369 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:56.883369 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:56.883369 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:56.883369 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:56.883369 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:28:56.883369 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:28:56.883369 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:28:56.883369 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:28:56.883369 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:28:56.883369 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:28:56.883369 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 16:28:56.883369 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:28:56.883369 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 310/607] 10.222.0.36 () {56 vars in 1102 bytes} [Thu May 19 16:28:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:56.884129 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466dfcb-f140-4957-9091-32388ebc3efe None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:56.917384 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61898bd4-3ae6-47de-a68c-65be5065639f None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:56.962417 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc914ec-932d-4726-920e-a05c7600f164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:56.963650 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc914ec-932d-4726-920e-a05c7600f164 None admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:56.964171 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc914ec-932d-4726-920e-a05c7600f164 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:56.964649 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc914ec-932d-4726-920e-a05c7600f164 None admin] PATH_INFO: `/v3/projects/631c76794a124d6da22c626e590e4352/users/33472643b65e4fe08fd5b3aa2c618133/roles/727026b648e1429ca3697e04800b328a` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:56.978648 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:56.991174 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc914ec-932d-4726-920e-a05c7600f164 None admin] RBAC: Authorizing `identity:create_grant(project_id=631c76794a124d6da22c626e590e4352, user_id=33472643b65e4fe08fd5b3aa2c618133, role_id=727026b648e1429ca3697e04800b328a)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:56.993988 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc914ec-932d-4726-920e-a05c7600f164 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:56.995730 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466dfcb-f140-4957-9091-32388ebc3efe None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:57.014372 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 298/608] 10.222.0.36 () {58 vars in 1439 bytes} [Thu May 19 16:28:56 2022] PUT /identity/v3/projects/631c76794a124d6da22c626e590e4352/users/33472643b65e4fe08fd5b3aa2c618133/roles/727026b648e1429ca3697e04800b328a => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:28:57.016028 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc914ec-932d-4726-920e-a05c7600f164 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:57.039418 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abfa1bb-7b42-4b5c-8ed4-32943d2d196f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:57.040335 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abfa1bb-7b42-4b5c-8ed4-32943d2d196f None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:57.040752 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abfa1bb-7b42-4b5c-8ed4-32943d2d196f None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:57.041171 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abfa1bb-7b42-4b5c-8ed4-32943d2d196f None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:57.042305 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abfa1bb-7b42-4b5c-8ed4-32943d2d196f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:57.042750 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abfa1bb-7b42-4b5c-8ed4-32943d2d196f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:57.044714 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abfa1bb-7b42-4b5c-8ed4-32943d2d196f None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:57.075574 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 311/609] 10.222.0.36 () {56 vars in 1148 bytes} [Thu May 19 16:28:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:57.081392 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abfa1bb-7b42-4b5c-8ed4-32943d2d196f None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:57.092092 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:57.149304 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc914ec-932d-4726-920e-a05c7600f164 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:57.194336 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6d2a5c-afc5-4965-aeec-2b47cb551c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:57.195515 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d2a5c-afc5-4965-aeec-2b47cb551c17 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:57.196026 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d2a5c-afc5-4965-aeec-2b47cb551c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:57.196524 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d2a5c-afc5-4965-aeec-2b47cb551c17 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:57.197604 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6d2a5c-afc5-4965-aeec-2b47cb551c17 None admin] RBAC: Adding query filter params (name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:57.198161 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6d2a5c-afc5-4965-aeec-2b47cb551c17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:57.200502 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6d2a5c-afc5-4965-aeec-2b47cb551c17 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:57.212249 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 299/610] 10.222.0.36 () {56 vars in 1103 bytes} [Thu May 19 16:28:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:57.218647 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6d2a5c-afc5-4965-aeec-2b47cb551c17 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:57.276675 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abfa1bb-7b42-4b5c-8ed4-32943d2d196f None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:57.322956 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d28c8a-3f97-4fb8-b3c7-d310ec7d8fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:57.323886 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d28c8a-3f97-4fb8-b3c7-d310ec7d8fe9 None admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:57.324172 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d28c8a-3f97-4fb8-b3c7-d310ec7d8fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:57.324448 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d28c8a-3f97-4fb8-b3c7-d310ec7d8fe9 None admin] PATH_INFO: `/v3/projects/631c76794a124d6da22c626e590e4352/users/2cc96d859a344042a4251a4104419855/roles/727026b648e1429ca3697e04800b328a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:57.341996 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d28c8a-3f97-4fb8-b3c7-d310ec7d8fe9 None admin] RBAC: Authorizing `identity:create_grant(project_id=631c76794a124d6da22c626e590e4352, user_id=2cc96d859a344042a4251a4104419855, role_id=727026b648e1429ca3697e04800b328a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:57.343580 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d28c8a-3f97-4fb8-b3c7-d310ec7d8fe9 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:57.348345 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:57.365261 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 312/611] 10.222.0.36 () {58 vars in 1439 bytes} [Thu May 19 16:28:57 2022] PUT /identity/v3/projects/631c76794a124d6da22c626e590e4352/users/2cc96d859a344042a4251a4104419855/roles/727026b648e1429ca3697e04800b328a => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:28:57.400908 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6d2a5c-afc5-4965-aeec-2b47cb551c17 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:57.427332 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:57.445366 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c55d1e-5475-4db6-81f8-7b39441fc3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:57.446644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c55d1e-5475-4db6-81f8-7b39441fc3c1 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:57.447134 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c55d1e-5475-4db6-81f8-7b39441fc3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:57.447636 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c55d1e-5475-4db6-81f8-7b39441fc3c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:57.461681 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c55d1e-5475-4db6-81f8-7b39441fc3c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:57.463725 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c55d1e-5475-4db6-81f8-7b39441fc3c1 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:57.494804 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-e5c55d1e-5475-4db6-81f8-7b39441fc3c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:57.494804 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:57.494804 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:57.494804 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:57.494804 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:57.494804 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:57.494804 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:57.494804 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:57.494804 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:57.494804 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:57.494804 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:57.494804 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:57.494804 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:28:57.494804 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:28:57.494804 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:28:57.495740 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:28:57.495740 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:57.495740 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:57.495740 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:57.495740 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:57.495740 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:57.495740 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:57.495740 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:57.495740 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:57.495740 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:57.495740 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:57.495740 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:57.495740 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:57.495740 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:57.495740 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:57.495740 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:57.495740 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:57.496647 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:28:57.496647 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:28:57.496647 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:57.496647 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:57.498733 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 300/612] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:28:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:57.504642 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d28c8a-3f97-4fb8-b3c7-d310ec7d8fe9 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:57.660247 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c5d940-63c1-40fe-a9a0-31ef97410f2f None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:57.660929 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c5d940-63c1-40fe-a9a0-31ef97410f2f None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:57.661476 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c5d940-63c1-40fe-a9a0-31ef97410f2f None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:57.663334 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 301/613] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:57.665008 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d28c8a-3f97-4fb8-b3c7-d310ec7d8fe9 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:57.672802 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b59658b-2f91-43eb-87ee-10e5a34ac0de None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:57.673103 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b59658b-2f91-43eb-87ee-10e5a34ac0de None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:57.673434 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b59658b-2f91-43eb-87ee-10e5a34ac0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:57.694173 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-8b59658b-2f91-43eb-87ee-10e5a34ac0de None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:57.695693 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-8b59658b-2f91-43eb-87ee-10e5a34ac0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:57.715910 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598e8d4f-59ce-4e25-a1f3-2c2c3c09e27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:57.717016 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e8d4f-59ce-4e25-a1f3-2c2c3c09e27d None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:57.717016 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e8d4f-59ce-4e25-a1f3-2c2c3c09e27d None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:57.717349 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e8d4f-59ce-4e25-a1f3-2c2c3c09e27d None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:57.718107 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598e8d4f-59ce-4e25-a1f3-2c2c3c09e27d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:57.718427 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598e8d4f-59ce-4e25-a1f3-2c2c3c09e27d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:57.720792 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598e8d4f-59ce-4e25-a1f3-2c2c3c09e27d None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:57.736461 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 313/614] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:28:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:57.742633 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598e8d4f-59ce-4e25-a1f3-2c2c3c09e27d None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:57.778425 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 302/615] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 16:28:57.819047 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b59658b-2f91-43eb-87ee-10e5a34ac0de None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:57.843563 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:57.924453 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598e8d4f-59ce-4e25-a1f3-2c2c3c09e27d None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:57.971500 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041243af-b339-44d0-822f-d9a2ff498e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:57.972319 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041243af-b339-44d0-822f-d9a2ff498e71 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:57.972611 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041243af-b339-44d0-822f-d9a2ff498e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:57.972893 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041243af-b339-44d0-822f-d9a2ff498e71 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:57.985282 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041243af-b339-44d0-822f-d9a2ff498e71 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:57.994928 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041243af-b339-44d0-822f-d9a2ff498e71 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:58.001563 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:58.005430 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b59658b-2f91-43eb-87ee-10e5a34ac0de None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:58.022659 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-041243af-b339-44d0-822f-d9a2ff498e71 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 16:28:58.022659 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:58.022659 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:58.022659 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:58.022659 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:58.022659 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:58.022659 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:58.022659 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:58.022659 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:58.022659 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:58.022659 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:58.022659 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:58.022659 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 16:28:58.022659 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 16:28:58.022659 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 16:28:58.022659 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 16:28:58.023680 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:58.023680 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:58.023680 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 16:28:58.023680 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:28:58.023680 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 16:28:58.023680 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:28:58.023680 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:58.023680 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:58.023680 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:58.023680 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:58.023680 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:58.023680 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:28:58.023680 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:58.023680 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:58.023680 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:58.023680 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:58.023680 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:58.023680 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:58.024612 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:58.024612 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:58.024612 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 16:28:58.024612 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 16:28:58.024612 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 16:28:58.024612 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 16:28:58.024612 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 16:28:58.024612 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 16:28:58.024612 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 16:28:58.024612 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:28:58.024612 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 314/616] 10.222.0.36 () {56 vars in 1096 bytes} [Thu May 19 16:28:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 16:28:58.029798 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041243af-b339-44d0-822f-d9a2ff498e71 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:58.059757 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e9f2c-f584-4d77-8872-99c211d4eb50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:58.060954 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e9f2c-f584-4d77-8872-99c211d4eb50 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:58.061466 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e9f2c-f584-4d77-8872-99c211d4eb50 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:58.061997 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e9f2c-f584-4d77-8872-99c211d4eb50 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:58.089173 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52e9f2c-f584-4d77-8872-99c211d4eb50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:28:58.097689 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-e52e9f2c-f584-4d77-8872-99c211d4eb50 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 16:28:58.097689 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:58.097689 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:58.097689 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:58.097689 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:58.097689 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:58.097689 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:58.097689 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:58.097689 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:58.097689 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:58.097689 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:58.097689 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:58.097689 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:28:58.097689 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:28:58.097689 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:28:58.097689 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:28:58.098663 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:28:58.098663 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:28:58.098663 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:28:58.098663 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:28:58.098663 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:58.098663 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:58.098663 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:58.098663 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:58.098663 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:58.098663 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:58.098663 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:58.098663 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:58.098663 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:58.098663 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:58.098663 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:58.098663 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:58.099517 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:58.099517 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:58.099517 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:58.099517 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:58.099517 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:28:58.099517 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:28:58.099517 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:28:58.099517 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:28:58.099517 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:28:58.099517 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:28:58.099517 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 16:28:58.099517 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:58.099517 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 303/617] 10.222.0.36 () {56 vars in 1102 bytes} [Thu May 19 16:28:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:58.106135 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e9f2c-f584-4d77-8872-99c211d4eb50 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:58.144190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041243af-b339-44d0-822f-d9a2ff498e71 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:58.162795 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:58.196579 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e52f360-29c5-4bce-9a57-e3c7c795daf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:58.197296 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e52f360-29c5-4bce-9a57-e3c7c795daf0 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:58.197586 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e52f360-29c5-4bce-9a57-e3c7c795daf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:58.197867 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e52f360-29c5-4bce-9a57-e3c7c795daf0 None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:58.198727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e52f360-29c5-4bce-9a57-e3c7c795daf0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:58.199124 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e52f360-29c5-4bce-9a57-e3c7c795daf0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:58.200675 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e52f360-29c5-4bce-9a57-e3c7c795daf0 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:58.238034 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 315/618] 10.222.0.36 () {56 vars in 1135 bytes} [Thu May 19 16:28:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:58.245311 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e52f360-29c5-4bce-9a57-e3c7c795daf0 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:58.321137 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e9f2c-f584-4d77-8872-99c211d4eb50 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:58.392351 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab557b9-5ae5-4e02-b3ac-76e09242f2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:58.393458 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab557b9-5ae5-4e02-b3ac-76e09242f2b2 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:58.393950 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab557b9-5ae5-4e02-b3ac-76e09242f2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:58.394427 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab557b9-5ae5-4e02-b3ac-76e09242f2b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:58.395663 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab557b9-5ae5-4e02-b3ac-76e09242f2b2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:58.396212 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab557b9-5ae5-4e02-b3ac-76e09242f2b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:58.398217 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab557b9-5ae5-4e02-b3ac-76e09242f2b2 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:58.413533 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 304/619] 10.222.0.36 () {56 vars in 1103 bytes} [Thu May 19 16:28:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:28:58.422192 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab557b9-5ae5-4e02-b3ac-76e09242f2b2 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:58.434696 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e52f360-29c5-4bce-9a57-e3c7c795daf0 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:58.477181 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:58.477381 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6869bf1-ba9b-4d1b-ba8d-75d3eeb29af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:58.477381 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6869bf1-ba9b-4d1b-ba8d-75d3eeb29af6 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:58.477381 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6869bf1-ba9b-4d1b-ba8d-75d3eeb29af6 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:58.477381 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6869bf1-ba9b-4d1b-ba8d-75d3eeb29af6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:58.486817 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6869bf1-ba9b-4d1b-ba8d-75d3eeb29af6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:58.488359 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6869bf1-ba9b-4d1b-ba8d-75d3eeb29af6 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:58.508312 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-b6869bf1-ba9b-4d1b-ba8d-75d3eeb29af6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:58.508312 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:58.508312 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:58.508312 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:58.508312 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:58.508312 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:58.508312 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:58.508312 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:58.508312 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:58.508312 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:58.508312 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:58.508312 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:58.508312 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:28:58.508312 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:28:58.508312 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:28:58.509288 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:28:58.509288 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:58.509288 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:58.509288 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:58.509288 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:58.509288 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:58.509288 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:58.509288 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:58.509288 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:28:58.509288 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:58.509288 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:58.509288 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:58.509288 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:58.509288 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:58.509288 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:58.509288 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:58.509288 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:58.510182 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:28:58.510182 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:28:58.510182 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:58.510182 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:28:58.510182 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 316/620] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:28:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:58.511245 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee6187d-655e-4e80-ae86-e0c09b7126dc None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:58.511539 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee6187d-655e-4e80-ae86-e0c09b7126dc None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:58.511821 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee6187d-655e-4e80-ae86-e0c09b7126dc None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:58.513387 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 317/621] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:58.516495 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee6187d-655e-4e80-ae86-e0c09b7126dc None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:58.548796 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:58.579850 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab557b9-5ae5-4e02-b3ac-76e09242f2b2 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:58.622616 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ab82df-f4ff-4195-8d9a-f67aae7d614b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:58.623790 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab82df-f4ff-4195-8d9a-f67aae7d614b None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:58.624285 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab82df-f4ff-4195-8d9a-f67aae7d614b None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:58.624893 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab82df-f4ff-4195-8d9a-f67aae7d614b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:58.637752 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ab82df-f4ff-4195-8d9a-f67aae7d614b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:58.639613 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ab82df-f4ff-4195-8d9a-f67aae7d614b None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:58.645443 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee6187d-655e-4e80-ae86-e0c09b7126dc None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:58.659199 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-46ab82df-f4ff-4195-8d9a-f67aae7d614b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:58.659199 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:58.659199 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:58.659199 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:58.659199 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:58.659199 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:58.659199 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:58.659199 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:58.659199 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:58.659199 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:58.659199 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:58.659199 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:58.659199 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:28:58.659199 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:28:58.659199 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:28:58.660144 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:28:58.660144 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:58.660144 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:58.660144 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:58.660144 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:58.660144 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:58.660144 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:58.660144 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:58.660144 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:58.660144 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:58.660144 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:58.660144 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:58.660144 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:58.660144 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:58.660144 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:58.660144 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:58.660144 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:58.661044 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:28:58.661044 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:28:58.661044 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:58.661044 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:58.663499 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 305/622] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:28:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:58.665038 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e6815-f721-4d85-b526-82a37dd9255e None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:58.665568 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e6815-f721-4d85-b526-82a37dd9255e None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:58.666049 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e6815-f721-4d85-b526-82a37dd9255e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:58.686829 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-0a1e6815-f721-4d85-b526-82a37dd9255e None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:28:58.688958 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-0a1e6815-f721-4d85-b526-82a37dd9255e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:28:58.693482 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71feb651-17dd-45b0-8cc0-57bd6cacffc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:58.694220 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71feb651-17dd-45b0-8cc0-57bd6cacffc3 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:58.694510 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71feb651-17dd-45b0-8cc0-57bd6cacffc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:58.694789 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71feb651-17dd-45b0-8cc0-57bd6cacffc3 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:58.695548 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71feb651-17dd-45b0-8cc0-57bd6cacffc3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:58.695862 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71feb651-17dd-45b0-8cc0-57bd6cacffc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:58.697373 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71feb651-17dd-45b0-8cc0-57bd6cacffc3 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:58.710723 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 318/623] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:28:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:58.711650 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71feb651-17dd-45b0-8cc0-57bd6cacffc3 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:58.781198 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 306/624] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:28:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:28:58.781198 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1e6815-f721-4d85-b526-82a37dd9255e None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:58.784938 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:58.829254 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:58.846578 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71feb651-17dd-45b0-8cc0-57bd6cacffc3 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:58.888108 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2f71b4-c775-4755-99eb-411356b6eee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:58.888800 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f71b4-c775-4755-99eb-411356b6eee3 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:58.889091 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f71b4-c775-4755-99eb-411356b6eee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:58.889407 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f71b4-c775-4755-99eb-411356b6eee3 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:58.890758 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2f71b4-c775-4755-99eb-411356b6eee3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:58.891271 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2f71b4-c775-4755-99eb-411356b6eee3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:58.892911 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2f71b4-c775-4755-99eb-411356b6eee3 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:58.902621 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1e6815-f721-4d85-b526-82a37dd9255e None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:58.910779 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 319/625] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:28:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:58.911639 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2f71b4-c775-4755-99eb-411356b6eee3 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:58.946647 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b90428c-c8c4-4b51-9689-cd329b85a030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:58.947752 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b90428c-c8c4-4b51-9689-cd329b85a030 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:58.948076 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b90428c-c8c4-4b51-9689-cd329b85a030 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:58.948359 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b90428c-c8c4-4b51-9689-cd329b85a030 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:58.958554 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b90428c-c8c4-4b51-9689-cd329b85a030 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:58.960013 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b90428c-c8c4-4b51-9689-cd329b85a030 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:58.979852 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-3b90428c-c8c4-4b51-9689-cd329b85a030 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:28:58.979852 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:58.979852 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:58.979852 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:58.979852 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:58.979852 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:58.979852 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:58.979852 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:58.979852 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:58.979852 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:58.979852 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:58.979852 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:58.979852 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 16:28:58.979852 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 16:28:58.979852 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 16:28:58.980810 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 16:28:58.980810 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:58.980810 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:58.980810 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:58.980810 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:58.980810 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:58.980810 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:58.980810 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:58.980810 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:58.980810 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:58.980810 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:58.980810 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:58.980810 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:58.980810 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:58.980810 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:58.980810 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:58.980810 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:58.981729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 16:28:58.981729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 16:28:58.981729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 16:28:58.981729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 16:28:58.981729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 16:28:58.981729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 16:28:58.981729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:28:58.981729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:58.981729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 307/626] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:28:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:58.982702 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b90428c-c8c4-4b51-9689-cd329b85a030 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:59.048695 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2f71b4-c775-4755-99eb-411356b6eee3 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:59.095776 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:59.121038 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e2570d-a3f2-4a49-b77f-684e9a809dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:59.123040 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e2570d-a3f2-4a49-b77f-684e9a809dd0 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:59.123529 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e2570d-a3f2-4a49-b77f-684e9a809dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:59.123998 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e2570d-a3f2-4a49-b77f-684e9a809dd0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:59.149723 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e2570d-a3f2-4a49-b77f-684e9a809dd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:28:59.161080 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b90428c-c8c4-4b51-9689-cd329b85a030 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:59.164099 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-f6e2570d-a3f2-4a49-b77f-684e9a809dd0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 16:28:59.164099 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:59.164099 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:59.164099 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:59.164099 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:59.164099 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:59.164099 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:59.164099 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:59.164099 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:59.164099 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:59.164099 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:59.164099 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:59.164099 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:28:59.164099 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:28:59.164099 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:28:59.164099 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:28:59.165045 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:28:59.165045 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:28:59.165045 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:28:59.165045 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:28:59.165045 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:59.165045 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:59.165045 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:59.165045 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:59.165045 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:59.165045 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:59.165045 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:59.165045 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:28:59.165045 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:59.165045 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:59.165045 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:59.165045 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:59.166026 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:59.166026 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:59.166026 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:59.166026 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:59.166026 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:28:59.166026 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:28:59.166026 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:28:59.166026 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:28:59.166026 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:28:59.166026 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:28:59.166026 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 16:28:59.166026 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:28:59.167113 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 320/627] 10.222.0.36 () {56 vars in 1102 bytes} [Thu May 19 16:28:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:59.171279 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e2570d-a3f2-4a49-b77f-684e9a809dd0 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:59.179099 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:59.201023 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97131ac-785a-43ec-8600-70f51ae69979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:59.201965 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97131ac-785a-43ec-8600-70f51ae69979 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:59.202319 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97131ac-785a-43ec-8600-70f51ae69979 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:59.202663 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97131ac-785a-43ec-8600-70f51ae69979 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:59.211445 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97131ac-785a-43ec-8600-70f51ae69979 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:59.213004 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97131ac-785a-43ec-8600-70f51ae69979 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:59.228792 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-f97131ac-785a-43ec-8600-70f51ae69979 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 16:28:59.228792 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:59.228792 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:59.228792 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:59.228792 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:59.228792 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:59.228792 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:59.228792 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:59.228792 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:59.228792 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:59.228792 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:59.228792 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:59.228792 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 16:28:59.228792 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 16:28:59.228792 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 16:28:59.228792 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 16:28:59.230899 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:59.230899 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:59.230899 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 16:28:59.230899 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:28:59.230899 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 16:28:59.230899 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:28:59.230899 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:59.230899 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:59.230899 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:59.230899 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:59.230899 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:59.230899 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:28:59.230899 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:59.230899 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:59.230899 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:59.230899 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:59.230899 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:59.230899 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:59.233218 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:59.233218 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:59.233218 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 16:28:59.233218 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 16:28:59.233218 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 16:28:59.233218 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 16:28:59.233218 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 16:28:59.233218 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 16:28:59.233218 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 16:28:59.233218 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:28:59.241180 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 308/628] 10.222.0.36 () {56 vars in 1093 bytes} [Thu May 19 16:28:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:59.241180 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97131ac-785a-43ec-8600-70f51ae69979 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:59.285020 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e2570d-a3f2-4a49-b77f-684e9a809dd0 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:59.323576 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24353bbe-44a7-4b88-9965-ae0f7e64e76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:59.324245 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24353bbe-44a7-4b88-9965-ae0f7e64e76d None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:59.324534 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24353bbe-44a7-4b88-9965-ae0f7e64e76d None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:59.324807 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24353bbe-44a7-4b88-9965-ae0f7e64e76d None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:59.325652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24353bbe-44a7-4b88-9965-ae0f7e64e76d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:59.326061 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24353bbe-44a7-4b88-9965-ae0f7e64e76d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:59.327672 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24353bbe-44a7-4b88-9965-ae0f7e64e76d None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:59.342175 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 321/629] 10.222.0.36 () {56 vars in 1148 bytes} [Thu May 19 16:28:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:59.343036 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24353bbe-44a7-4b88-9965-ae0f7e64e76d None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:59.346984 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97131ac-785a-43ec-8600-70f51ae69979 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:59.372418 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:59.385548 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24248391-0ccd-48d4-9443-dba4595fb3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:59.386222 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24248391-0ccd-48d4-9443-dba4595fb3cc None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:59.386511 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24248391-0ccd-48d4-9443-dba4595fb3cc None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:59.386796 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24248391-0ccd-48d4-9443-dba4595fb3cc None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:59.387566 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24248391-0ccd-48d4-9443-dba4595fb3cc None admin] RBAC: Adding query filter params (name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:59.387878 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24248391-0ccd-48d4-9443-dba4595fb3cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:59.389305 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24248391-0ccd-48d4-9443-dba4595fb3cc None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:59.398960 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 309/630] 10.222.0.36 () {56 vars in 1103 bytes} [Thu May 19 16:28:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:59.399909 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24248391-0ccd-48d4-9443-dba4595fb3cc None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:59.421901 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:59.461835 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24353bbe-44a7-4b88-9965-ae0f7e64e76d None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:59.500925 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c51b1cb-2ffe-4479-adef-3be14c4ff034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:59.502203 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51b1cb-2ffe-4479-adef-3be14c4ff034 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:59.502669 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51b1cb-2ffe-4479-adef-3be14c4ff034 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:59.503159 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51b1cb-2ffe-4479-adef-3be14c4ff034 None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:59.504280 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c51b1cb-2ffe-4479-adef-3be14c4ff034 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:59.504782 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c51b1cb-2ffe-4479-adef-3be14c4ff034 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:59.506588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c51b1cb-2ffe-4479-adef-3be14c4ff034 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:59.511066 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24248391-0ccd-48d4-9443-dba4595fb3cc None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:59.527406 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 322/631] 10.222.0.36 () {56 vars in 1133 bytes} [Thu May 19 16:28:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:28:59.528740 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c51b1cb-2ffe-4479-adef-3be14c4ff034 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:59.556344 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3296d2b4-cfa1-404e-80d2-6f36e871f68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:59.557352 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3296d2b4-cfa1-404e-80d2-6f36e871f68e None admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:59.557785 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3296d2b4-cfa1-404e-80d2-6f36e871f68e None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:59.558072 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3296d2b4-cfa1-404e-80d2-6f36e871f68e None admin] PATH_INFO: `/v3/projects/631c76794a124d6da22c626e590e4352/users/ffe7cebc6f3b40d8a8e1866ed2da9822/roles/727026b648e1429ca3697e04800b328a` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:59.574853 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3296d2b4-cfa1-404e-80d2-6f36e871f68e None admin] RBAC: Authorizing `identity:create_grant(project_id=631c76794a124d6da22c626e590e4352, user_id=ffe7cebc6f3b40d8a8e1866ed2da9822, role_id=727026b648e1429ca3697e04800b328a)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:59.576399 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3296d2b4-cfa1-404e-80d2-6f36e871f68e None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:59.595922 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 310/632] 10.222.0.36 () {58 vars in 1439 bytes} [Thu May 19 16:28:59 2022] PUT /identity/v3/projects/631c76794a124d6da22c626e590e4352/users/ffe7cebc6f3b40d8a8e1866ed2da9822/roles/727026b648e1429ca3697e04800b328a => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:28:59.597422 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3296d2b4-cfa1-404e-80d2-6f36e871f68e None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:59.655798 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c51b1cb-2ffe-4479-adef-3be14c4ff034 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:59.674914 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:59.694283 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:28:59.706779 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfbc7fb-0f4e-4c9b-8ee9-1f3bbdf138e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:59.708300 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfbc7fb-0f4e-4c9b-8ee9-1f3bbdf138e1 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:59.708824 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfbc7fb-0f4e-4c9b-8ee9-1f3bbdf138e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:59.709313 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfbc7fb-0f4e-4c9b-8ee9-1f3bbdf138e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:59.723750 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfbc7fb-0f4e-4c9b-8ee9-1f3bbdf138e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:59.725677 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfbc7fb-0f4e-4c9b-8ee9-1f3bbdf138e1 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:59.749568 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-bcfbc7fb-0f4e-4c9b-8ee9-1f3bbdf138e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:59.749568 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:28:59.749568 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:28:59.749568 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:28:59.749568 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:28:59.749568 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:28:59.749568 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:28:59.749568 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:28:59.749568 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:28:59.749568 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:28:59.749568 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:28:59.749568 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:28:59.749568 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:28:59.749568 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:28:59.749568 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:28:59.750514 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:28:59.750514 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:28:59.750514 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:28:59.750514 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:28:59.750514 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:28:59.750514 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:28:59.750514 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:28:59.750514 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:28:59.750514 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:28:59.750514 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:28:59.750514 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:28:59.750514 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:28:59.750514 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:28:59.750514 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:28:59.750514 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:28:59.750514 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:28:59.750514 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:28:59.751440 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:28:59.751440 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:28:59.751440 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:28:59.751440 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:28:59.754476 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 323/633] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:28:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:28:59.760261 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfbc7fb-0f4e-4c9b-8ee9-1f3bbdf138e1 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:28:59.844073 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3296d2b4-cfa1-404e-80d2-6f36e871f68e None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:59.886726 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfbc7fb-0f4e-4c9b-8ee9-1f3bbdf138e1 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:28:59.921024 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375fc97c-26ad-4e14-adb6-ccb9ac4ff988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:59.924545 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375fc97c-26ad-4e14-adb6-ccb9ac4ff988 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:59.924545 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375fc97c-26ad-4e14-adb6-ccb9ac4ff988 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:59.924545 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375fc97c-26ad-4e14-adb6-ccb9ac4ff988 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:59.946160 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1778b3ba-bc8d-4535-afb0-f1f68ab51fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:28:59.946831 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778b3ba-bc8d-4535-afb0-f1f68ab51fc9 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:59.947120 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778b3ba-bc8d-4535-afb0-f1f68ab51fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:59.947401 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778b3ba-bc8d-4535-afb0-f1f68ab51fc9 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:59.948156 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1778b3ba-bc8d-4535-afb0-f1f68ab51fc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:28:59.948471 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1778b3ba-bc8d-4535-afb0-f1f68ab51fc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:28:59.949959 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1778b3ba-bc8d-4535-afb0-f1f68ab51fc9 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:28:59.965768 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 324/634] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:28:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:28:59.967169 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25bcbf6-0cf5-463d-acf7-ce0d22b82cc2 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:28:59.967465 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25bcbf6-0cf5-463d-acf7-ce0d22b82cc2 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:28:59.967749 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25bcbf6-0cf5-463d-acf7-ce0d22b82cc2 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:28:59.969990 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 325/635] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:28:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:28:59.975054 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25bcbf6-0cf5-463d-acf7-ce0d22b82cc2 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:00.024113 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:00.045772 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375fc97c-26ad-4e14-adb6-ccb9ac4ff988 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:00.048078 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375fc97c-26ad-4e14-adb6-ccb9ac4ff988 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:00.103258 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25bcbf6-0cf5-463d-acf7-ce0d22b82cc2 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:00.104588 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:00.160049 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bfdf68-b72b-4e27-b772-8e205823f62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:00.160859 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bfdf68-b72b-4e27-b772-8e205823f62f None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:00.161141 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bfdf68-b72b-4e27-b772-8e205823f62f None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:00.161567 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bfdf68-b72b-4e27-b772-8e205823f62f None admin] PATH_INFO: `/v3/users/glance` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:00.169556 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bfdf68-b72b-4e27-b772-8e205823f62f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:00.171090 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bfdf68-b72b-4e27-b772-8e205823f62f None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:00.191725 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-71bfdf68-b72b-4e27-b772-8e205823f62f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 16:29:00.191725 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:00.191725 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:00.191725 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:00.191725 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:00.191725 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:00.191725 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:00.191725 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:00.191725 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:00.191725 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:00.191725 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:00.191725 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:00.191725 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 16:29:00.191725 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 16:29:00.191725 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 16:29:00.191725 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 16:29:00.197331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:00.197331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:00.197331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 16:29:00.197331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:29:00.197331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 16:29:00.197331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:29:00.197331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:00.197331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:00.197331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:00.197331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:00.197331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:00.197331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:00.197331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:00.197331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:00.197331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:00.197331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:00.197331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:00.197331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:00.199506 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:00.199506 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:00.199506 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 16:29:00.199506 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 16:29:00.199506 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 16:29:00.199506 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 16:29:00.199506 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 16:29:00.199506 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 16:29:00.199506 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 16:29:00.199506 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:00.199506 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 326/636] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:28:59 2022] GET /identity/v3/users/glance => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:00.199506 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c7701-8d69-49b8-826c-db256dfe4962 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:00.199506 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c7701-8d69-49b8-826c-db256dfe4962 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:00.201277 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c7701-8d69-49b8-826c-db256dfe4962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:00.201486 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 311/637] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:28:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 602 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 16:29:00.201486 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80db8cc-27c8-4c61-8e8c-76b621f0edcf None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:00.201486 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80db8cc-27c8-4c61-8e8c-76b621f0edcf None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:00.201486 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80db8cc-27c8-4c61-8e8c-76b621f0edcf None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:00.204351 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 312/638] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:29:00.204351 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80db8cc-27c8-4c61-8e8c-76b621f0edcf None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:00.221009 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-e86c7701-8d69-49b8-826c-db256dfe4962 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:00.222516 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e86c7701-8d69-49b8-826c-db256dfe4962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:00.309388 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 327/639] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:29:00.309388 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86c7701-8d69-49b8-826c-db256dfe4962 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:00.331325 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80db8cc-27c8-4c61-8e8c-76b621f0edcf None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:00.340604 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:00.368941 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa18388-5d43-4ea0-a509-7aace405f7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:00.369649 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa18388-5d43-4ea0-a509-7aace405f7c3 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:00.369936 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa18388-5d43-4ea0-a509-7aace405f7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:00.370215 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa18388-5d43-4ea0-a509-7aace405f7c3 None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:00.371192 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa18388-5d43-4ea0-a509-7aace405f7c3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:00.371487 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa18388-5d43-4ea0-a509-7aace405f7c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:00.373092 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa18388-5d43-4ea0-a509-7aace405f7c3 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:00.375807 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:00.391494 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 313/640] 10.222.0.36 () {56 vars in 1137 bytes} [Thu May 19 16:29:00 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:00.392612 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6dcae5-05cd-45ae-a22c-e6f31f6c742b None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:00.392900 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6dcae5-05cd-45ae-a22c-e6f31f6c742b None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:00.393192 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6dcae5-05cd-45ae-a22c-e6f31f6c742b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:00.413065 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-8b6dcae5-05cd-45ae-a22c-e6f31f6c742b None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:00.414311 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-8b6dcae5-05cd-45ae-a22c-e6f31f6c742b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:00.432290 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86c7701-8d69-49b8-826c-db256dfe4962 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:00.475965 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea6bd01-8228-46b2-b0f4-08438f817b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:00.476885 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea6bd01-8228-46b2-b0f4-08438f817b75 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:00.477352 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea6bd01-8228-46b2-b0f4-08438f817b75 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:00.477637 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea6bd01-8228-46b2-b0f4-08438f817b75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:00.489297 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 314/641] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:29:00.490478 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6dcae5-05cd-45ae-a22c-e6f31f6c742b None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:00.496606 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea6bd01-8228-46b2-b0f4-08438f817b75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:00.498183 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea6bd01-8228-46b2-b0f4-08438f817b75 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:00.536770 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-8ea6bd01-8228-46b2-b0f4-08438f817b75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:00.536770 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:00.536770 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:00.536770 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:00.536770 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:00.536770 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:00.536770 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:00.536770 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:00.536770 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:00.536770 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:00.536770 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:00.536770 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:00.536770 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:29:00.536770 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:29:00.536770 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:29:00.537747 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:29:00.537747 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:00.537747 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:00.537747 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:00.537747 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:00.537747 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:00.537747 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:00.537747 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:00.537747 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:00.537747 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:00.537747 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:00.537747 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:00.537747 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:00.537747 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:00.537747 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:00.537747 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:00.537747 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:00.538629 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:29:00.538629 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:29:00.538629 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:00.538629 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:00.543302 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 328/642] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:29:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:00.544189 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea6bd01-8228-46b2-b0f4-08438f817b75 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:00.634690 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6dcae5-05cd-45ae-a22c-e6f31f6c742b None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:00.651000 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:00.667396 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:00.681955 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea6bd01-8228-46b2-b0f4-08438f817b75 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:00.682844 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d070a8a-73d6-430a-afd0-d2d8b6406f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:00.683805 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d070a8a-73d6-430a-afd0-d2d8b6406f1c None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:00.684143 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d070a8a-73d6-430a-afd0-d2d8b6406f1c None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:00.685421 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d070a8a-73d6-430a-afd0-d2d8b6406f1c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:00.700664 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d070a8a-73d6-430a-afd0-d2d8b6406f1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:29:00.709390 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-0d070a8a-73d6-430a-afd0-d2d8b6406f1c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 16:29:00.709390 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:00.709390 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:00.709390 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:00.709390 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:00.709390 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:00.709390 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:00.709390 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:00.709390 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:00.709390 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:00.709390 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:00.709390 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:00.709390 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:29:00.709390 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:29:00.709390 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:29:00.709390 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:29:00.710377 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:29:00.710377 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:29:00.710377 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:29:00.710377 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:29:00.710377 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:00.710377 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:00.710377 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:00.710377 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:00.710377 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:00.710377 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:00.710377 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:00.710377 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:00.710377 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:00.710377 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:00.710377 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:00.710377 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:00.711220 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:00.711220 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:00.711220 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:00.711220 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:00.711220 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:29:00.711220 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:29:00.711220 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:29:00.711220 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:29:00.711220 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:29:00.711220 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:29:00.711220 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 16:29:00.711220 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:00.715318 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 315/643] 10.222.0.36 () {56 vars in 1102 bytes} [Thu May 19 16:29:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:00.716050 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d070a8a-73d6-430a-afd0-d2d8b6406f1c None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:00.729200 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf5f0ac-1b8d-4809-a391-a7ce5648cbbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:00.730256 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf5f0ac-1b8d-4809-a391-a7ce5648cbbc None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:00.730880 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf5f0ac-1b8d-4809-a391-a7ce5648cbbc None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:00.731226 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf5f0ac-1b8d-4809-a391-a7ce5648cbbc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:00.811606 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf5f0ac-1b8d-4809-a391-a7ce5648cbbc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:00.813518 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf5f0ac-1b8d-4809-a391-a7ce5648cbbc None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:00.831949 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d070a8a-73d6-430a-afd0-d2d8b6406f1c None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:00.882254 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d09a817-2df7-4c2d-932f-d58c878e9763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:00.883060 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d09a817-2df7-4c2d-932f-d58c878e9763 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:00.883342 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d09a817-2df7-4c2d-932f-d58c878e9763 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:00.883619 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d09a817-2df7-4c2d-932f-d58c878e9763 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:00.901051 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d09a817-2df7-4c2d-932f-d58c878e9763 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:29:00.911600 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:00.913141 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-1d09a817-2df7-4c2d-932f-d58c878e9763 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 16:29:00.913141 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:00.913141 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:00.913141 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:00.913141 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:00.913141 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:00.913141 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:00.913141 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:00.913141 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:00.913141 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:00.913141 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:00.913141 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:00.913141 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:29:00.913141 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:29:00.913141 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:29:00.913141 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:29:00.914333 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:29:00.914333 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:29:00.914333 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:29:00.914333 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:29:00.914333 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:00.914333 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:00.914333 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:00.914333 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:00.914333 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:00.914333 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:00.914333 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:00.914333 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:00.914333 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:00.914333 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:00.914333 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:00.914333 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:00.915419 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:00.915419 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:00.915419 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:00.915419 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:00.915419 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:29:00.915419 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:29:00.915419 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:29:00.915419 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:29:00.915419 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:29:00.915419 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:29:00.915419 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 16:29:00.915419 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:00.915419 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 316/644] 10.222.0.36 () {56 vars in 1102 bytes} [Thu May 19 16:29:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:00.916144 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d09a817-2df7-4c2d-932f-d58c878e9763 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:00.951983 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 329/645] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:29:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 16:29:00.953223 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf5f0ac-1b8d-4809-a391-a7ce5648cbbc None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:00.970352 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:01.031230 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d09a817-2df7-4c2d-932f-d58c878e9763 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:01.056662 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf5f0ac-1b8d-4809-a391-a7ce5648cbbc None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:01.070238 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659cde05-2741-4c28-9ca9-11c00bb18ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:01.071208 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659cde05-2741-4c28-9ca9-11c00bb18ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:01.071639 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659cde05-2741-4c28-9ca9-11c00bb18ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:01.072063 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659cde05-2741-4c28-9ca9-11c00bb18ad7 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:01.072919 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659cde05-2741-4c28-9ca9-11c00bb18ad7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:01.073412 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659cde05-2741-4c28-9ca9-11c00bb18ad7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:01.075440 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659cde05-2741-4c28-9ca9-11c00bb18ad7 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:01.088617 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 317/646] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:29:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:01.089685 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659cde05-2741-4c28-9ca9-11c00bb18ad7 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:01.098273 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7f0ad1-fcf1-40fd-9d69-f9111c031e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:01.098958 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f0ad1-fcf1-40fd-9d69-f9111c031e3f None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:01.099251 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f0ad1-fcf1-40fd-9d69-f9111c031e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:01.099542 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f0ad1-fcf1-40fd-9d69-f9111c031e3f None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:01.100310 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7f0ad1-fcf1-40fd-9d69-f9111c031e3f None admin] RBAC: Adding query filter params (name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:01.100627 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7f0ad1-fcf1-40fd-9d69-f9111c031e3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:01.102078 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7f0ad1-fcf1-40fd-9d69-f9111c031e3f None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:01.116365 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 330/647] 10.222.0.36 () {56 vars in 1103 bytes} [Thu May 19 16:29:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:01.116365 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7f0ad1-fcf1-40fd-9d69-f9111c031e3f None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:01.160414 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:01.206751 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659cde05-2741-4c28-9ca9-11c00bb18ad7 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:01.210726 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:01.225763 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7f0ad1-fcf1-40fd-9d69-f9111c031e3f None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:01.252586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda5aa76-ba31-4460-9c57-25c8f60fc94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:01.253315 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5aa76-ba31-4460-9c57-25c8f60fc94f None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:01.253605 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5aa76-ba31-4460-9c57-25c8f60fc94f None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:01.253931 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5aa76-ba31-4460-9c57-25c8f60fc94f None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:01.254639 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda5aa76-ba31-4460-9c57-25c8f60fc94f None admin] RBAC: Adding query filter params (name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:01.254947 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda5aa76-ba31-4460-9c57-25c8f60fc94f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:01.256323 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda5aa76-ba31-4460-9c57-25c8f60fc94f None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:01.260853 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d7590d-f1b4-4acd-ac25-8c0ae9bfa71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:01.261989 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7590d-f1b4-4acd-ac25-8c0ae9bfa71a None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:01.261989 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7590d-f1b4-4acd-ac25-8c0ae9bfa71a None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:01.262276 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7590d-f1b4-4acd-ac25-8c0ae9bfa71a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:01.270798 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 318/648] 10.222.0.36 () {56 vars in 1103 bytes} [Thu May 19 16:29:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:01.270798 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda5aa76-ba31-4460-9c57-25c8f60fc94f None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:01.277079 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d7590d-f1b4-4acd-ac25-8c0ae9bfa71a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:01.278592 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d7590d-f1b4-4acd-ac25-8c0ae9bfa71a None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:01.296312 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-62d7590d-f1b4-4acd-ac25-8c0ae9bfa71a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:01.296312 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:01.296312 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:01.296312 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:01.296312 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:01.296312 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:01.296312 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:01.296312 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:01.296312 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:01.296312 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:01.296312 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:01.296312 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:01.296312 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:29:01.296312 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:29:01.296312 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:29:01.297258 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:29:01.297258 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:01.297258 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:01.297258 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:01.297258 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:01.297258 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:01.297258 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:01.297258 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:01.297258 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:01.297258 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:01.297258 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:01.297258 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:01.297258 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:01.297258 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:01.297258 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:01.297258 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:01.297258 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:01.298287 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:29:01.298287 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:29:01.298287 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:01.298287 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:01.298287 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 331/649] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:29:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:01.298562 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d7590d-f1b4-4acd-ac25-8c0ae9bfa71a None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:01.380016 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda5aa76-ba31-4460-9c57-25c8f60fc94f None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:01.404144 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:01.415529 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d7590d-f1b4-4acd-ac25-8c0ae9bfa71a None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:01.424121 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00684bb4-1b77-41dc-a724-cd04fb971894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:01.425014 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00684bb4-1b77-41dc-a724-cd04fb971894 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:01.425321 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00684bb4-1b77-41dc-a724-cd04fb971894 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:01.425601 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00684bb4-1b77-41dc-a724-cd04fb971894 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:01.433297 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00684bb4-1b77-41dc-a724-cd04fb971894 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:01.434753 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00684bb4-1b77-41dc-a724-cd04fb971894 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:01.449571 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-00684bb4-1b77-41dc-a724-cd04fb971894 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:29:01.449571 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:01.449571 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:01.449571 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:01.449571 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:01.449571 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:01.449571 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:01.449571 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:01.449571 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:01.449571 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:01.449571 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:01.449571 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:01.449571 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 16:29:01.449571 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 16:29:01.449571 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 16:29:01.450492 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 16:29:01.450492 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:01.450492 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:01.450492 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:01.450492 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:01.450492 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:01.450492 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:01.450492 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:01.450492 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:01.450492 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:01.450492 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:01.450492 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:01.450492 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:01.450492 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:01.450492 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:01.450492 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:01.450492 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:01.451367 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 16:29:01.451367 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 16:29:01.451367 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 16:29:01.451367 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 16:29:01.451367 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 16:29:01.451367 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 16:29:01.451367 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:29:01.451367 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:01.451367 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 319/650] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:29:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:01.451855 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00684bb4-1b77-41dc-a724-cd04fb971894 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:01.460230 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c916181-27e7-4971-833e-af499311953d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:01.460398 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c916181-27e7-4971-833e-af499311953d None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:01.460994 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c916181-27e7-4971-833e-af499311953d None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:01.460994 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c916181-27e7-4971-833e-af499311953d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:01.473615 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c916181-27e7-4971-833e-af499311953d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:01.475073 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c916181-27e7-4971-833e-af499311953d None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:01.481586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:01.493561 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-9c916181-27e7-4971-833e-af499311953d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:01.493561 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:01.493561 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:01.493561 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:01.493561 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:01.493561 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:01.493561 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:01.493561 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:01.493561 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:01.493561 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:01.493561 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:01.493561 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:01.493561 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:29:01.493561 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:29:01.493561 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:29:01.494536 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:29:01.494536 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:01.494536 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:01.494536 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:01.494536 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:01.494536 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:01.494536 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:01.494536 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:01.494536 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:01.494536 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:01.494536 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:01.494536 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:01.494536 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:01.494536 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:01.494536 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:01.494536 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:01.494536 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:01.495445 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:29:01.495445 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:29:01.495445 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:01.495445 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:01.495445 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 332/651] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:29:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:01.496314 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c916181-27e7-4971-833e-af499311953d None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:01.566238 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00684bb4-1b77-41dc-a724-cd04fb971894 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:01.603111 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385fc5bc-5e15-466a-ab51-eeffa1a5e201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:01.604180 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385fc5bc-5e15-466a-ab51-eeffa1a5e201 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:01.604464 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385fc5bc-5e15-466a-ab51-eeffa1a5e201 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:01.604741 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385fc5bc-5e15-466a-ab51-eeffa1a5e201 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:01.607046 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c916181-27e7-4971-833e-af499311953d None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:01.616183 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385fc5bc-5e15-466a-ab51-eeffa1a5e201 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:01.617686 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385fc5bc-5e15-466a-ab51-eeffa1a5e201 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:01.636858 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-385fc5bc-5e15-466a-ab51-eeffa1a5e201 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:01.636858 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:01.636858 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:01.636858 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:01.636858 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:01.636858 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:01.636858 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:01.636858 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:01.636858 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:01.636858 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:01.636858 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:01.636858 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:01.636858 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:29:01.636858 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:29:01.636858 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:29:01.637817 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:29:01.637817 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:01.637817 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:01.637817 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:01.637817 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:01.637817 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:01.637817 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:01.637817 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:01.637817 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:01.637817 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:01.637817 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:01.637817 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:01.637817 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:01.637817 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:01.637817 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:01.637817 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:01.637817 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:01.638791 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:29:01.638791 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:29:01.638791 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:01.638791 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:01.638791 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 320/652] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:29:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:01.639062 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385fc5bc-5e15-466a-ab51-eeffa1a5e201 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:01.651561 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7267b20f-88e4-4658-bea7-009582a2f50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:01.652550 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7267b20f-88e4-4658-bea7-009582a2f50f None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:01.652963 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7267b20f-88e4-4658-bea7-009582a2f50f None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:01.653405 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7267b20f-88e4-4658-bea7-009582a2f50f None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:01.654143 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7267b20f-88e4-4658-bea7-009582a2f50f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:01.654454 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7267b20f-88e4-4658-bea7-009582a2f50f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:01.656612 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7267b20f-88e4-4658-bea7-009582a2f50f None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:01.670727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 333/653] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:29:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:01.671850 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7267b20f-88e4-4658-bea7-009582a2f50f None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:01.676456 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:01.728057 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:01.753848 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385fc5bc-5e15-466a-ab51-eeffa1a5e201 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:01.784010 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7267b20f-88e4-4658-bea7-009582a2f50f None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:01.794689 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4a1d03-7b66-47ed-9e7e-31674c660d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:01.795373 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a1d03-7b66-47ed-9e7e-31674c660d05 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:01.796019 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a1d03-7b66-47ed-9e7e-31674c660d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:01.796305 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a1d03-7b66-47ed-9e7e-31674c660d05 None admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:01.797511 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4a1d03-7b66-47ed-9e7e-31674c660d05 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:01.797939 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4a1d03-7b66-47ed-9e7e-31674c660d05 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:01.799291 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4a1d03-7b66-47ed-9e7e-31674c660d05 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:01.811553 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 321/654] 10.222.0.36 () {56 vars in 1148 bytes} [Thu May 19 16:29:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:01.812386 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4a1d03-7b66-47ed-9e7e-31674c660d05 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:01.827406 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e42e04-897d-4514-98c7-6b2787fe35cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:01.828397 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e42e04-897d-4514-98c7-6b2787fe35cc None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:01.829486 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e42e04-897d-4514-98c7-6b2787fe35cc None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:01.829983 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e42e04-897d-4514-98c7-6b2787fe35cc None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:01.830922 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e42e04-897d-4514-98c7-6b2787fe35cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:01.831419 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e42e04-897d-4514-98c7-6b2787fe35cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:01.832919 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e42e04-897d-4514-98c7-6b2787fe35cc None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:01.850077 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 334/655] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:29:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:01.851089 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e42e04-897d-4514-98c7-6b2787fe35cc None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:01.927259 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4a1d03-7b66-47ed-9e7e-31674c660d05 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:01.936281 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:01.963840 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37029ba1-6b07-4877-91be-f79d8a69aedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:01.964526 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37029ba1-6b07-4877-91be-f79d8a69aedd None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:01.964813 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37029ba1-6b07-4877-91be-f79d8a69aedd None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:01.965098 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37029ba1-6b07-4877-91be-f79d8a69aedd None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:01.965870 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37029ba1-6b07-4877-91be-f79d8a69aedd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:01.966182 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37029ba1-6b07-4877-91be-f79d8a69aedd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:01.967571 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37029ba1-6b07-4877-91be-f79d8a69aedd None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:01.975467 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:01.985434 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 322/656] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:29:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:01.985434 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37029ba1-6b07-4877-91be-f79d8a69aedd None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:01.985683 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e42e04-897d-4514-98c7-6b2787fe35cc None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:02.025179 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16bc131-975c-468c-a87d-d88b16953685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:02.026120 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16bc131-975c-468c-a87d-d88b16953685 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:02.026881 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16bc131-975c-468c-a87d-d88b16953685 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:02.027166 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16bc131-975c-468c-a87d-d88b16953685 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:02.035450 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16bc131-975c-468c-a87d-d88b16953685 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:02.037756 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16bc131-975c-468c-a87d-d88b16953685 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:02.052777 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-e16bc131-975c-468c-a87d-d88b16953685 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:29:02.052777 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:02.052777 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:02.052777 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:02.052777 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:02.052777 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:02.052777 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:02.052777 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:02.052777 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:02.052777 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:02.052777 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:02.052777 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:02.052777 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 16:29:02.052777 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 16:29:02.052777 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 16:29:02.053839 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 16:29:02.053839 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:02.053839 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:02.053839 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:02.053839 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:02.053839 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:02.053839 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:02.053839 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:02.053839 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:02.053839 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:02.053839 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:02.053839 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:02.053839 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:02.053839 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:02.053839 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:02.053839 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:02.053839 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:02.054723 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 16:29:02.054723 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 16:29:02.054723 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 16:29:02.054723 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 16:29:02.054723 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 16:29:02.054723 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 16:29:02.054723 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:29:02.054723 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:02.054723 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 335/657] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:29:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:02.055204 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16bc131-975c-468c-a87d-d88b16953685 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:02.089383 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37029ba1-6b07-4877-91be-f79d8a69aedd None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:02.127437 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72197469-f838-4fb2-81d6-c5653b4e518b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:02.128089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72197469-f838-4fb2-81d6-c5653b4e518b None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:02.128370 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72197469-f838-4fb2-81d6-c5653b4e518b None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:02.128642 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72197469-f838-4fb2-81d6-c5653b4e518b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:02.129524 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72197469-f838-4fb2-81d6-c5653b4e518b None admin] RBAC: Adding query filter params (role.id=727026b648e1429ca3697e04800b328a, scope.project.id=631c76794a124d6da22c626e590e4352, user.id=58bdeada00b74b7b8e186e6dc51e6dfb) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:02.129834 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72197469-f838-4fb2-81d6-c5653b4e518b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:02.131257 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72197469-f838-4fb2-81d6-c5653b4e518b None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:02.137087 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 323/658] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:29:01 2022] GET /identity/v3/role_assignments?user.id=58bdeada00b74b7b8e186e6dc51e6dfb&scope.project.id=631c76794a124d6da22c626e590e4352&role.id=727026b648e1429ca3697e04800b328a => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:02.137892 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72197469-f838-4fb2-81d6-c5653b4e518b None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:02.160790 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16bc131-975c-468c-a87d-d88b16953685 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:02.189954 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:02.203869 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973de082-9d15-43e7-9226-0a4b79ed9083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:02.204702 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973de082-9d15-43e7-9226-0a4b79ed9083 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:02.204988 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973de082-9d15-43e7-9226-0a4b79ed9083 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:02.205302 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973de082-9d15-43e7-9226-0a4b79ed9083 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:02.214374 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973de082-9d15-43e7-9226-0a4b79ed9083 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:02.215923 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973de082-9d15-43e7-9226-0a4b79ed9083 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:02.219743 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:02.232095 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-973de082-9d15-43e7-9226-0a4b79ed9083 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 16:29:02.232095 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:02.232095 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:02.232095 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:02.232095 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:02.232095 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:02.232095 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:02.232095 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:02.232095 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:02.232095 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:02.232095 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:02.232095 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:02.232095 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 16:29:02.232095 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 16:29:02.232095 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 16:29:02.232095 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 16:29:02.233356 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:02.233356 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:02.233356 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 16:29:02.233356 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:29:02.233356 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 16:29:02.233356 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:29:02.233356 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:02.233356 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:02.233356 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:02.233356 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:02.233356 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:02.233356 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:02.233356 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:02.233356 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:02.233356 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:02.233356 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:02.233356 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:02.233356 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:02.234472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:02.234472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:02.234472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 16:29:02.234472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 16:29:02.234472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 16:29:02.234472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 16:29:02.234472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 16:29:02.234472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 16:29:02.234472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 16:29:02.234472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:02.234472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 336/659] 10.222.0.36 () {56 vars in 1102 bytes} [Thu May 19 16:29:02 2022] GET /identity/v3/users/neutron => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:02.236359 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503da13-97a6-4888-99b1-6d11ba23cd89 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:02.236630 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503da13-97a6-4888-99b1-6d11ba23cd89 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:02.236950 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503da13-97a6-4888-99b1-6d11ba23cd89 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:02.243520 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 337/660] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:29:02.243520 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b503da13-97a6-4888-99b1-6d11ba23cd89 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:02.271949 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72197469-f838-4fb2-81d6-c5653b4e518b None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:02.310928 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d4a598-f43b-4481-a3f1-bdbd885a715d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:02.311746 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4a598-f43b-4481-a3f1-bdbd885a715d None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:02.312026 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4a598-f43b-4481-a3f1-bdbd885a715d None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:02.312299 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4a598-f43b-4481-a3f1-bdbd885a715d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:02.321370 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d4a598-f43b-4481-a3f1-bdbd885a715d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:02.322848 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d4a598-f43b-4481-a3f1-bdbd885a715d None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:02.337849 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-f5d4a598-f43b-4481-a3f1-bdbd885a715d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 16:29:02.337849 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:02.337849 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:02.337849 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:02.337849 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:02.337849 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:02.337849 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:02.337849 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:02.337849 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:02.337849 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:02.337849 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:02.337849 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:02.337849 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 16:29:02.337849 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 16:29:02.337849 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 16:29:02.337849 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 16:29:02.338857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:02.338857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:02.338857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 16:29:02.338857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:29:02.338857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 16:29:02.338857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:29:02.338857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:02.338857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:02.338857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:02.338857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:02.338857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:02.338857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:02.338857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:02.338857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:02.338857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:02.338857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:02.338857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:02.338857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:02.340071 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:02.340071 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:02.340071 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 16:29:02.340071 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 16:29:02.340071 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 16:29:02.340071 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 16:29:02.340071 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 16:29:02.340071 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 16:29:02.340071 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 16:29:02.340071 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:02.340071 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 324/661] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:29:02 2022] GET /identity/v3/users/cinder => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:02.340071 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d4a598-f43b-4481-a3f1-bdbd885a715d None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:02.362430 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b503da13-97a6-4888-99b1-6d11ba23cd89 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:02.403252 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85021b12-3f38-42e8-b936-1e26d798fc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:02.404103 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85021b12-3f38-42e8-b936-1e26d798fc8c None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:02.404492 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85021b12-3f38-42e8-b936-1e26d798fc8c None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:02.404875 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85021b12-3f38-42e8-b936-1e26d798fc8c None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:02.405836 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85021b12-3f38-42e8-b936-1e26d798fc8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:02.406263 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85021b12-3f38-42e8-b936-1e26d798fc8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:02.407837 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85021b12-3f38-42e8-b936-1e26d798fc8c None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:02.420846 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 338/662] 10.222.0.36 () {56 vars in 1148 bytes} [Thu May 19 16:29:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:02.422202 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46efd04e-1cb3-413f-8560-f49218e9815b None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:02.423008 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46efd04e-1cb3-413f-8560-f49218e9815b None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:02.423320 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46efd04e-1cb3-413f-8560-f49218e9815b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:02.441762 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-46efd04e-1cb3-413f-8560-f49218e9815b None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:02.442963 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-46efd04e-1cb3-413f-8560-f49218e9815b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:02.463423 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d4a598-f43b-4481-a3f1-bdbd885a715d None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:02.488190 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:02.492723 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:02.502326 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645415ad-9c18-4422-8611-4677bef8dbd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:02.503098 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645415ad-9c18-4422-8611-4677bef8dbd1 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:02.503457 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645415ad-9c18-4422-8611-4677bef8dbd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:02.503827 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645415ad-9c18-4422-8611-4677bef8dbd1 None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:02.504755 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645415ad-9c18-4422-8611-4677bef8dbd1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:02.505139 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645415ad-9c18-4422-8611-4677bef8dbd1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:02.506700 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645415ad-9c18-4422-8611-4677bef8dbd1 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:02.520817 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 339/663] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:29:02.521700 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46efd04e-1cb3-413f-8560-f49218e9815b None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:02.527137 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 325/664] 10.222.0.36 () {56 vars in 1139 bytes} [Thu May 19 16:29:02 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:02.528265 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645415ad-9c18-4422-8611-4677bef8dbd1 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:02.651423 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645415ad-9c18-4422-8611-4677bef8dbd1 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:02.655562 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46efd04e-1cb3-413f-8560-f49218e9815b None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:02.689260 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6cd02e-b67b-4749-a274-c40d161c9475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:02.689926 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6cd02e-b67b-4749-a274-c40d161c9475 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:02.690214 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6cd02e-b67b-4749-a274-c40d161c9475 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:02.690497 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6cd02e-b67b-4749-a274-c40d161c9475 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:02.691335 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6cd02e-b67b-4749-a274-c40d161c9475 None admin] RBAC: Adding query filter params (role.id=727026b648e1429ca3697e04800b328a, scope.project.id=631c76794a124d6da22c626e590e4352, user.id=57a5d7151478434f86be085bcc487f0c) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:02.691656 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6cd02e-b67b-4749-a274-c40d161c9475 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:02.693088 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6cd02e-b67b-4749-a274-c40d161c9475 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:02.696500 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78624b53-96c0-46e0-a885-884a332c63bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:02.697649 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78624b53-96c0-46e0-a885-884a332c63bd None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:02.698103 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78624b53-96c0-46e0-a885-884a332c63bd None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:02.698531 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78624b53-96c0-46e0-a885-884a332c63bd None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:02.699115 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 326/665] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:29:02 2022] GET /identity/v3/role_assignments?user.id=57a5d7151478434f86be085bcc487f0c&scope.project.id=631c76794a124d6da22c626e590e4352&role.id=727026b648e1429ca3697e04800b328a => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:02.699674 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78624b53-96c0-46e0-a885-884a332c63bd None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:02.699922 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6cd02e-b67b-4749-a274-c40d161c9475 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:02.700343 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78624b53-96c0-46e0-a885-884a332c63bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:02.701843 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78624b53-96c0-46e0-a885-884a332c63bd None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:02.733198 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 340/666] 10.222.0.36 () {56 vars in 1137 bytes} [Thu May 19 16:29:02 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:02.733198 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78624b53-96c0-46e0-a885-884a332c63bd None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:02.771667 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:02.784898 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:02.859931 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6cd02e-b67b-4749-a274-c40d161c9475 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:02.861767 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78624b53-96c0-46e0-a885-884a332c63bd None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:02.907599 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391a2a9b-fd47-4beb-b68b-1706c88f457d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:02.908630 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a2a9b-fd47-4beb-b68b-1706c88f457d None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:02.909177 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a2a9b-fd47-4beb-b68b-1706c88f457d None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:02.909770 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a2a9b-fd47-4beb-b68b-1706c88f457d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:02.912959 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd18e8c2-05ee-4b9c-a2d7-a415ca125722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:02.914520 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18e8c2-05ee-4b9c-a2d7-a415ca125722 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:02.914520 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18e8c2-05ee-4b9c-a2d7-a415ca125722 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:02.916516 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18e8c2-05ee-4b9c-a2d7-a415ca125722 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:02.932482 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd18e8c2-05ee-4b9c-a2d7-a415ca125722 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:29:02.940574 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-cd18e8c2-05ee-4b9c-a2d7-a415ca125722 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 16:29:02.940574 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:02.940574 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:02.940574 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:02.940574 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:02.940574 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:02.940574 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:02.940574 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:02.940574 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:02.940574 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:02.940574 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:02.940574 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:02.940574 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:29:02.940574 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:29:02.940574 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:29:02.940574 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:29:02.941590 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:29:02.941590 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:29:02.941590 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:29:02.941590 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:29:02.941590 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:02.941590 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:02.941590 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:02.941590 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:02.941590 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:02.941590 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:02.941590 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:02.941590 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:02.941590 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:02.941590 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:02.941590 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:02.941590 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:02.942439 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:02.942439 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:02.942439 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:02.942439 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:02.942439 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:29:02.942439 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:29:02.942439 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:29:02.942439 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:29:02.942439 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:29:02.942439 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:29:02.942439 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 16:29:02.942439 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:02.942439 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 341/667] 10.222.0.36 () {56 vars in 1102 bytes} [Thu May 19 16:29:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:02.943146 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd18e8c2-05ee-4b9c-a2d7-a415ca125722 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:03.006592 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391a2a9b-fd47-4beb-b68b-1706c88f457d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:03.008398 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391a2a9b-fd47-4beb-b68b-1706c88f457d None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:03.064039 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:03.074021 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd18e8c2-05ee-4b9c-a2d7-a415ca125722 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:03.097537 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:03.110908 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5a93ac-8e39-4846-8ada-d161e81b17cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:03.111941 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a93ac-8e39-4846-8ada-d161e81b17cf None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:03.112247 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a93ac-8e39-4846-8ada-d161e81b17cf None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:03.112557 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a93ac-8e39-4846-8ada-d161e81b17cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:03.120461 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 327/668] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:29:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 16:29:03.121367 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391a2a9b-fd47-4beb-b68b-1706c88f457d None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:03.193261 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5a93ac-8e39-4846-8ada-d161e81b17cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:03.194760 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5a93ac-8e39-4846-8ada-d161e81b17cf None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:03.233627 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391a2a9b-fd47-4beb-b68b-1706c88f457d None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:03.271558 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dc244f-665d-46cc-87bd-c2d0b318bf78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:03.272618 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc244f-665d-46cc-87bd-c2d0b318bf78 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:03.273182 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc244f-665d-46cc-87bd-c2d0b318bf78 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:03.273659 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc244f-665d-46cc-87bd-c2d0b318bf78 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:03.274601 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dc244f-665d-46cc-87bd-c2d0b318bf78 None admin] RBAC: Adding query filter params (name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:03.275173 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dc244f-665d-46cc-87bd-c2d0b318bf78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:03.276860 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dc244f-665d-46cc-87bd-c2d0b318bf78 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:03.286441 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 328/669] 10.222.0.36 () {56 vars in 1103 bytes} [Thu May 19 16:29:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:03.287655 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dc244f-665d-46cc-87bd-c2d0b318bf78 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:03.289906 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:03.305062 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 342/670] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:29:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 16:29:03.306065 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5a93ac-8e39-4846-8ada-d161e81b17cf None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:03.309570 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:03.408962 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dc244f-665d-46cc-87bd-c2d0b318bf78 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:03.414609 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5a93ac-8e39-4846-8ada-d161e81b17cf None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:03.448040 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdcd018-0435-4154-b529-21c4b910ea39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:03.448899 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdcd018-0435-4154-b529-21c4b910ea39 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:03.449213 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdcd018-0435-4154-b529-21c4b910ea39 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:03.449500 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdcd018-0435-4154-b529-21c4b910ea39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:03.453084 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59597e36-bbb7-45b1-8e2a-35453e3a84c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:03.454649 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59597e36-bbb7-45b1-8e2a-35453e3a84c8 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:03.455105 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59597e36-bbb7-45b1-8e2a-35453e3a84c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:03.455527 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59597e36-bbb7-45b1-8e2a-35453e3a84c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:03.461027 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdcd018-0435-4154-b529-21c4b910ea39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:03.462547 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdcd018-0435-4154-b529-21c4b910ea39 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:03.468661 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59597e36-bbb7-45b1-8e2a-35453e3a84c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:03.470857 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59597e36-bbb7-45b1-8e2a-35453e3a84c8 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:03.481697 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-3fdcd018-0435-4154-b529-21c4b910ea39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:03.481697 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:03.481697 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:03.481697 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:03.481697 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:03.481697 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:03.481697 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:03.481697 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:03.481697 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:03.481697 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:03.481697 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:03.481697 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:03.481697 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:29:03.481697 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:29:03.481697 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:29:03.482801 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:29:03.482801 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:03.482801 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:03.482801 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:03.482801 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:03.482801 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:03.482801 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:03.482801 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:03.482801 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:03.482801 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:03.482801 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:03.482801 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:03.482801 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:03.482801 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:03.482801 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:03.482801 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:03.482801 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:03.483912 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:29:03.483912 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:29:03.483912 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:03.483912 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:03.487529 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 329/671] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:29:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:03.488461 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdcd018-0435-4154-b529-21c4b910ea39 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:03.495697 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-59597e36-bbb7-45b1-8e2a-35453e3a84c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:03.495697 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:03.495697 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:03.495697 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:03.495697 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:03.495697 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:03.495697 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:03.495697 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:03.495697 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:03.495697 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:03.495697 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:03.495697 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:03.495697 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:29:03.495697 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:29:03.495697 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:29:03.496732 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:29:03.496732 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:03.496732 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:03.496732 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:03.496732 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:03.496732 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:03.496732 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:03.496732 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:03.496732 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:03.496732 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:03.496732 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:03.496732 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:03.496732 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:03.496732 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:03.496732 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:03.496732 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:03.496732 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:03.498197 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:29:03.498197 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:29:03.498197 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:03.498197 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:03.498197 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 343/672] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:29:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:03.498197 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59597e36-bbb7-45b1-8e2a-35453e3a84c8 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:03.563968 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:03.570821 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:03.618309 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdcd018-0435-4154-b529-21c4b910ea39 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:03.637236 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59597e36-bbb7-45b1-8e2a-35453e3a84c8 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:03.665431 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be451eb-9c89-4d04-bf31-4bdf5d1d0cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:03.666282 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be451eb-9c89-4d04-bf31-4bdf5d1d0cfb None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:03.666599 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be451eb-9c89-4d04-bf31-4bdf5d1d0cfb None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:03.667099 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be451eb-9c89-4d04-bf31-4bdf5d1d0cfb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:03.678662 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e851377-56d0-4069-a572-7acba302feef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:03.679558 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e851377-56d0-4069-a572-7acba302feef None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:03.679951 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e851377-56d0-4069-a572-7acba302feef None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:03.680319 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e851377-56d0-4069-a572-7acba302feef None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:03.680650 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be451eb-9c89-4d04-bf31-4bdf5d1d0cfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:03.681282 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e851377-56d0-4069-a572-7acba302feef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:03.681686 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e851377-56d0-4069-a572-7acba302feef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:03.682196 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be451eb-9c89-4d04-bf31-4bdf5d1d0cfb None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:03.683260 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e851377-56d0-4069-a572-7acba302feef None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:03.696132 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 344/673] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:29:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:03.697109 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e851377-56d0-4069-a572-7acba302feef None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:03.709629 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-0be451eb-9c89-4d04-bf31-4bdf5d1d0cfb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:03.709629 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:03.709629 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:03.709629 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:03.709629 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:03.709629 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:03.709629 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:03.709629 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:03.709629 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:03.709629 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:03.709629 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:03.709629 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:03.709629 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:29:03.709629 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:29:03.709629 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:29:03.711291 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:29:03.711291 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:03.711291 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:03.711291 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:03.711291 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:03.711291 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:03.711291 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:03.711291 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:03.711291 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:03.711291 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:03.711291 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:03.711291 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:03.711291 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:03.711291 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:03.711291 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:03.711291 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:03.711291 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:03.712410 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:29:03.712410 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:29:03.712410 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:03.712410 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:03.712410 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 330/674] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:29:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:03.712410 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be451eb-9c89-4d04-bf31-4bdf5d1d0cfb None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:03.834274 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e851377-56d0-4069-a572-7acba302feef None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:03.837660 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:03.849260 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be451eb-9c89-4d04-bf31-4bdf5d1d0cfb None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:03.860971 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:03.884453 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79565d0-d1fe-4f0e-a46b-a666e4cd45a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:03.885088 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79565d0-d1fe-4f0e-a46b-a666e4cd45a3 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:03.885423 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79565d0-d1fe-4f0e-a46b-a666e4cd45a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:03.885705 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79565d0-d1fe-4f0e-a46b-a666e4cd45a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:03.886441 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79565d0-d1fe-4f0e-a46b-a666e4cd45a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:03.886752 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79565d0-d1fe-4f0e-a46b-a666e4cd45a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:03.888159 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79565d0-d1fe-4f0e-a46b-a666e4cd45a3 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:03.889922 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74db479-b21a-4491-898d-b8ff6905d978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:03.891604 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74db479-b21a-4491-898d-b8ff6905d978 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:03.893091 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74db479-b21a-4491-898d-b8ff6905d978 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:03.893805 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74db479-b21a-4491-898d-b8ff6905d978 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:03.903161 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74db479-b21a-4491-898d-b8ff6905d978 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:03.905134 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74db479-b21a-4491-898d-b8ff6905d978 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:03.907983 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 345/675] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:29:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:03.909595 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79565d0-d1fe-4f0e-a46b-a666e4cd45a3 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:03.925955 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-a74db479-b21a-4491-898d-b8ff6905d978 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:29:03.925955 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:03.925955 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:03.925955 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:03.925955 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:03.925955 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:03.925955 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:03.925955 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:03.925955 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:03.925955 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:03.925955 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:03.925955 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:03.925955 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 16:29:03.925955 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 16:29:03.925955 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 16:29:03.926878 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 16:29:03.926878 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:03.926878 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:03.926878 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:03.926878 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:03.926878 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:03.926878 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:03.926878 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:03.926878 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:03.926878 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:03.926878 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:03.926878 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:03.926878 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:03.926878 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:03.926878 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:03.926878 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:03.926878 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:03.927764 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 16:29:03.927764 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 16:29:03.927764 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 16:29:03.927764 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 16:29:03.927764 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 16:29:03.927764 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 16:29:03.927764 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:29:03.927764 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:03.930377 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 331/676] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:29:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:03.931392 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74db479-b21a-4491-898d-b8ff6905d978 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:04.051375 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74db479-b21a-4491-898d-b8ff6905d978 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:04.052338 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79565d0-d1fe-4f0e-a46b-a666e4cd45a3 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:04.100799 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b705d-6cbe-4b0c-98d4-5c5716298ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:04.101647 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b705d-6cbe-4b0c-98d4-5c5716298ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:04.101940 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b705d-6cbe-4b0c-98d4-5c5716298ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:04.102222 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b705d-6cbe-4b0c-98d4-5c5716298ce5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:04.103955 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c9616b-6e58-4360-b01e-8614c5397b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:04.104903 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9616b-6e58-4360-b01e-8614c5397b81 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:04.105519 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9616b-6e58-4360-b01e-8614c5397b81 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:04.105979 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9616b-6e58-4360-b01e-8614c5397b81 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:04.106870 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c9616b-6e58-4360-b01e-8614c5397b81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:04.107284 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c9616b-6e58-4360-b01e-8614c5397b81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:04.108782 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c9616b-6e58-4360-b01e-8614c5397b81 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:04.112308 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690b705d-6cbe-4b0c-98d4-5c5716298ce5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:04.113980 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690b705d-6cbe-4b0c-98d4-5c5716298ce5 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:04.122571 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 346/677] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:29:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:04.123695 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c9616b-6e58-4360-b01e-8614c5397b81 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:04.133144 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-690b705d-6cbe-4b0c-98d4-5c5716298ce5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 16:29:04.133144 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:04.133144 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:04.133144 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:04.133144 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:04.133144 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:04.133144 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:04.133144 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:04.133144 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:04.133144 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:04.133144 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:04.133144 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:04.133144 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 16:29:04.133144 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 16:29:04.133144 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 16:29:04.133144 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 16:29:04.134385 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:04.134385 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:04.134385 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 16:29:04.134385 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:29:04.134385 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 16:29:04.134385 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:29:04.134385 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:04.134385 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:04.134385 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:04.134385 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:04.134385 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:04.134385 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:04.134385 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:04.134385 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:04.134385 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:04.134385 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:04.134385 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:04.134385 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:04.135573 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:04.135573 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:04.135573 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 16:29:04.135573 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 16:29:04.135573 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 16:29:04.135573 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 16:29:04.135573 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 16:29:04.135573 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 16:29:04.135573 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 16:29:04.135573 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:04.138517 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 332/678] 10.222.0.36 () {56 vars in 1096 bytes} [Thu May 19 16:29:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:04.139502 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b705d-6cbe-4b0c-98d4-5c5716298ce5 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:04.160863 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:04.180860 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:04.260622 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c9616b-6e58-4360-b01e-8614c5397b81 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:04.263436 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b705d-6cbe-4b0c-98d4-5c5716298ce5 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:04.309953 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d0bdc0-bd02-41c5-8e2f-60a148c94b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:04.310764 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d0bdc0-bd02-41c5-8e2f-60a148c94b2f None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:04.311051 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d0bdc0-bd02-41c5-8e2f-60a148c94b2f None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:04.311328 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d0bdc0-bd02-41c5-8e2f-60a148c94b2f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:04.315152 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74230fbb-9d95-4179-9ca5-36244db640af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:04.315808 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74230fbb-9d95-4179-9ca5-36244db640af None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:04.316092 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74230fbb-9d95-4179-9ca5-36244db640af None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:04.316372 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74230fbb-9d95-4179-9ca5-36244db640af None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:04.317199 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74230fbb-9d95-4179-9ca5-36244db640af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:04.317522 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74230fbb-9d95-4179-9ca5-36244db640af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:04.318986 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74230fbb-9d95-4179-9ca5-36244db640af None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:04.324149 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d0bdc0-bd02-41c5-8e2f-60a148c94b2f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:04.325948 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d0bdc0-bd02-41c5-8e2f-60a148c94b2f None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:04.337472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 347/679] 10.222.0.36 () {56 vars in 1148 bytes} [Thu May 19 16:29:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:04.338494 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74230fbb-9d95-4179-9ca5-36244db640af None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:04.346107 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-62d0bdc0-bd02-41c5-8e2f-60a148c94b2f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:29:04.346107 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:04.346107 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:04.346107 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:04.346107 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:04.346107 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:04.346107 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:04.346107 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:04.346107 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:04.346107 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:04.346107 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:04.346107 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:04.346107 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 16:29:04.346107 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 16:29:04.346107 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 16:29:04.347063 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 16:29:04.347063 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:04.347063 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:04.347063 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:04.347063 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:04.347063 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:04.347063 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:04.347063 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:04.347063 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:04.347063 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:04.347063 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:04.347063 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:04.347063 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:04.347063 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:04.347063 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:04.347063 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:04.347063 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:04.348062 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 16:29:04.348062 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 16:29:04.348062 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 16:29:04.348062 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 16:29:04.348062 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 16:29:04.348062 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 16:29:04.348062 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:29:04.348062 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:04.351517 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 333/680] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:29:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:04.352934 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d0bdc0-bd02-41c5-8e2f-60a148c94b2f None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:04.461561 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:04.465171 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:04.478351 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d0bdc0-bd02-41c5-8e2f-60a148c94b2f None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:04.480647 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74230fbb-9d95-4179-9ca5-36244db640af None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:04.518217 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd5f3fe-2690-4741-af64-90fb896ac671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:04.518918 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4337268f-6499-4216-bd8a-28118d94e79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:04.519044 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd5f3fe-2690-4741-af64-90fb896ac671 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:04.519363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd5f3fe-2690-4741-af64-90fb896ac671 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:04.519716 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd5f3fe-2690-4741-af64-90fb896ac671 None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:04.520519 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4337268f-6499-4216-bd8a-28118d94e79e None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:04.520948 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd5f3fe-2690-4741-af64-90fb896ac671 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:04.521176 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4337268f-6499-4216-bd8a-28118d94e79e None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:04.521424 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd5f3fe-2690-4741-af64-90fb896ac671 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:04.521822 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4337268f-6499-4216-bd8a-28118d94e79e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:04.522841 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4337268f-6499-4216-bd8a-28118d94e79e None admin] RBAC: Adding query filter params (role.id=727026b648e1429ca3697e04800b328a, scope.project.id=631c76794a124d6da22c626e590e4352, user.id=33472643b65e4fe08fd5b3aa2c618133) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:04.523081 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd5f3fe-2690-4741-af64-90fb896ac671 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:04.523515 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4337268f-6499-4216-bd8a-28118d94e79e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:04.525292 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4337268f-6499-4216-bd8a-28118d94e79e None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:04.533066 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 334/681] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:29:04 2022] GET /identity/v3/role_assignments?user.id=33472643b65e4fe08fd5b3aa2c618133&scope.project.id=631c76794a124d6da22c626e590e4352&role.id=727026b648e1429ca3697e04800b328a => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:04.534267 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4337268f-6499-4216-bd8a-28118d94e79e None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:04.556606 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 348/682] 10.222.0.36 () {56 vars in 1135 bytes} [Thu May 19 16:29:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:04.556606 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c39b888-a02c-415f-a4bb-8870ce506dc8 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:04.556606 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c39b888-a02c-415f-a4bb-8870ce506dc8 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:04.556606 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c39b888-a02c-415f-a4bb-8870ce506dc8 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:04.556606 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 349/683] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:29:04.562993 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080cddb-e740-4a2b-842e-e4803ffa6610 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:04.563282 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080cddb-e740-4a2b-842e-e4803ffa6610 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:04.563559 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080cddb-e740-4a2b-842e-e4803ffa6610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:04.583647 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-0080cddb-e740-4a2b-842e-e4803ffa6610 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:04.584938 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-0080cddb-e740-4a2b-842e-e4803ffa6610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:04.662072 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4337268f-6499-4216-bd8a-28118d94e79e None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:04.677344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 350/684] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:29:04.678696 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0080cddb-e740-4a2b-842e-e4803ffa6610 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:04.720856 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2a01a5-1671-45c9-a586-b6ba4683c378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:04.721546 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a01a5-1671-45c9-a586-b6ba4683c378 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:04.721839 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a01a5-1671-45c9-a586-b6ba4683c378 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:04.722118 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a01a5-1671-45c9-a586-b6ba4683c378 None admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:04.722940 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2a01a5-1671-45c9-a586-b6ba4683c378 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:04.723257 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2a01a5-1671-45c9-a586-b6ba4683c378 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:04.724923 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2a01a5-1671-45c9-a586-b6ba4683c378 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:04.739041 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 335/685] 10.222.0.36 () {56 vars in 1148 bytes} [Thu May 19 16:29:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:04.740009 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2a01a5-1671-45c9-a586-b6ba4683c378 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:04.756865 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:04.785144 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:04.821038 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0080cddb-e740-4a2b-842e-e4803ffa6610 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:04.853491 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2a01a5-1671-45c9-a586-b6ba4683c378 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:04.860887 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56bf0b4-f332-4ba1-8f93-5e6f7f327825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:04.861959 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56bf0b4-f332-4ba1-8f93-5e6f7f327825 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:04.861959 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56bf0b4-f332-4ba1-8f93-5e6f7f327825 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:04.862160 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56bf0b4-f332-4ba1-8f93-5e6f7f327825 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:04.902899 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0013068e-c6b2-4af1-9ad6-2e2ef4c16704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:04.904657 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0013068e-c6b2-4af1-9ad6-2e2ef4c16704 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:04.905126 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0013068e-c6b2-4af1-9ad6-2e2ef4c16704 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:04.905589 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0013068e-c6b2-4af1-9ad6-2e2ef4c16704 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:04.920776 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0013068e-c6b2-4af1-9ad6-2e2ef4c16704 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:29:04.931560 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-0013068e-c6b2-4af1-9ad6-2e2ef4c16704 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:29:04.931560 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:04.931560 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:04.931560 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:04.931560 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:04.931560 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:04.931560 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:04.931560 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:04.931560 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:04.931560 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:04.931560 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:04.931560 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:04.931560 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:29:04.931560 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:29:04.931560 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:29:04.931560 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:29:04.932755 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:29:04.932755 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:29:04.932755 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:29:04.932755 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:29:04.932755 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:04.932755 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:04.932755 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:04.932755 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:04.932755 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:04.932755 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:04.932755 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:04.932755 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:04.932755 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:04.932755 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:04.932755 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:04.932755 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:04.933713 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:04.933713 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:04.933713 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:04.933713 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:04.933713 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:29:04.933713 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:29:04.933713 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:29:04.933713 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:29:04.933713 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:29:04.933713 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:29:04.933713 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:29:04.933713 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:04.935078 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 336/686] 10.222.0.36 () {56 vars in 1096 bytes} [Thu May 19 16:29:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:04.935749 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0013068e-c6b2-4af1-9ad6-2e2ef4c16704 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:04.956158 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56bf0b4-f332-4ba1-8f93-5e6f7f327825 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:04.957989 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56bf0b4-f332-4ba1-8f93-5e6f7f327825 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:05.048337 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:05.054262 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:05.084434 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0013068e-c6b2-4af1-9ad6-2e2ef4c16704 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:05.096480 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 351/687] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:29:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 16:29:05.097630 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56bf0b4-f332-4ba1-8f93-5e6f7f327825 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:05.127684 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71623322-9981-4343-93ec-5dc239268b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:05.128736 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71623322-9981-4343-93ec-5dc239268b87 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:05.129438 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71623322-9981-4343-93ec-5dc239268b87 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:05.130054 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71623322-9981-4343-93ec-5dc239268b87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:05.131146 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71623322-9981-4343-93ec-5dc239268b87 None admin] RBAC: Adding query filter params (role.id=727026b648e1429ca3697e04800b328a, scope.project.id=631c76794a124d6da22c626e590e4352, user.id=2cc96d859a344042a4251a4104419855) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:05.131792 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71623322-9981-4343-93ec-5dc239268b87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:05.133806 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71623322-9981-4343-93ec-5dc239268b87 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:05.141579 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 337/688] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:29:04 2022] GET /identity/v3/role_assignments?user.id=2cc96d859a344042a4251a4104419855&scope.project.id=631c76794a124d6da22c626e590e4352&role.id=727026b648e1429ca3697e04800b328a => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:05.142802 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71623322-9981-4343-93ec-5dc239268b87 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:05.211738 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56bf0b4-f332-4ba1-8f93-5e6f7f327825 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:05.248631 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab79e9-cf70-43b9-b815-a406179a30ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:05.249360 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab79e9-cf70-43b9-b815-a406179a30ce None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:05.249649 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab79e9-cf70-43b9-b815-a406179a30ce None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:05.249933 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab79e9-cf70-43b9-b815-a406179a30ce None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:05.251307 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ab79e9-cf70-43b9-b815-a406179a30ce None admin] RBAC: Adding query filter params (name=admin) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:05.251620 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ab79e9-cf70-43b9-b815-a406179a30ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:05.253358 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ab79e9-cf70-43b9-b815-a406179a30ce None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:05.263316 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 352/689] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:29:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:05.268598 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab79e9-cf70-43b9-b815-a406179a30ce None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:05.273980 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71623322-9981-4343-93ec-5dc239268b87 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:05.315138 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:05.318357 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da43d4e1-87a1-466e-9c60-a5af692babd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:05.319577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da43d4e1-87a1-466e-9c60-a5af692babd7 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:05.320108 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da43d4e1-87a1-466e-9c60-a5af692babd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:05.320599 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da43d4e1-87a1-466e-9c60-a5af692babd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:05.329526 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:05.335972 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da43d4e1-87a1-466e-9c60-a5af692babd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:05.337902 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da43d4e1-87a1-466e-9c60-a5af692babd7 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:05.358051 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-da43d4e1-87a1-466e-9c60-a5af692babd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:05.358051 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:05.358051 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:05.358051 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:05.358051 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:05.358051 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:05.358051 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:05.358051 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:05.358051 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:05.358051 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:05.358051 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:05.358051 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:05.358051 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:29:05.358051 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:29:05.358051 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:29:05.359017 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:29:05.359017 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:05.359017 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:05.359017 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:05.359017 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:05.359017 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:05.359017 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:05.359017 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:05.359017 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:05.359017 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:05.359017 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:05.359017 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:05.359017 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:05.359017 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:05.359017 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:05.359017 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:05.359017 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:05.359951 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:29:05.359951 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:29:05.359951 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:05.359951 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:05.359951 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 338/690] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:29:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:05.361546 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2604f2fd-4a82-4e1a-aab9-65afe82954eb None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:05.361902 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2604f2fd-4a82-4e1a-aab9-65afe82954eb None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:05.362249 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2604f2fd-4a82-4e1a-aab9-65afe82954eb None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:05.367705 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 339/691] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:29:05.367705 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2604f2fd-4a82-4e1a-aab9-65afe82954eb None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:05.419946 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab79e9-cf70-43b9-b815-a406179a30ce None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:05.471239 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2658ea80-7f9d-4a07-bf46-627c75fdbbca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:05.472687 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658ea80-7f9d-4a07-bf46-627c75fdbbca None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:05.473134 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658ea80-7f9d-4a07-bf46-627c75fdbbca None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:05.474503 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658ea80-7f9d-4a07-bf46-627c75fdbbca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:05.489003 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2658ea80-7f9d-4a07-bf46-627c75fdbbca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:05.490530 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2658ea80-7f9d-4a07-bf46-627c75fdbbca None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:05.498542 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2604f2fd-4a82-4e1a-aab9-65afe82954eb None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:05.511896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-2658ea80-7f9d-4a07-bf46-627c75fdbbca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:05.511896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:05.511896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:05.511896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:05.511896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:05.511896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:05.511896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:05.511896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:05.511896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:05.511896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:05.511896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:05.511896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:05.511896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:29:05.511896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:29:05.511896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:29:05.512918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:29:05.512918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:05.512918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:05.512918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:05.512918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:05.512918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:05.512918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:05.512918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:05.512918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:05.512918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:05.512918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:05.512918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:05.512918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:05.512918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:05.512918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:05.512918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:05.512918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:05.514562 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:29:05.514562 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:29:05.514562 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:05.514562 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:05.514562 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 353/692] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:29:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:05.514880 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9007f7-9593-411d-b56a-4ecdcc2b31fa None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:05.515136 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9007f7-9593-411d-b56a-4ecdcc2b31fa None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:05.515302 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9007f7-9593-411d-b56a-4ecdcc2b31fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:05.549065 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b317c01-0a4d-4f2c-ae36-18243a3b6f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:05.550259 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b317c01-0a4d-4f2c-ae36-18243a3b6f75 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:05.550751 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b317c01-0a4d-4f2c-ae36-18243a3b6f75 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:05.551265 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b317c01-0a4d-4f2c-ae36-18243a3b6f75 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:05.552325 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b317c01-0a4d-4f2c-ae36-18243a3b6f75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:05.552970 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b317c01-0a4d-4f2c-ae36-18243a3b6f75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:05.554768 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b317c01-0a4d-4f2c-ae36-18243a3b6f75 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:05.575376 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 340/693] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:29:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:05.576848 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b317c01-0a4d-4f2c-ae36-18243a3b6f75 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:05.639487 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:05.665911 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-dd9007f7-9593-411d-b56a-4ecdcc2b31fa None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:05.667087 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-dd9007f7-9593-411d-b56a-4ecdcc2b31fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:05.709055 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b317c01-0a4d-4f2c-ae36-18243a3b6f75 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:05.747485 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a89630-2650-465d-b1fe-88a7fcb0f917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:05.748123 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a89630-2650-465d-b1fe-88a7fcb0f917 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:05.748415 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a89630-2650-465d-b1fe-88a7fcb0f917 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:05.748700 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a89630-2650-465d-b1fe-88a7fcb0f917 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:05.749504 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a89630-2650-465d-b1fe-88a7fcb0f917 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:05.749819 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a89630-2650-465d-b1fe-88a7fcb0f917 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:05.751805 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a89630-2650-465d-b1fe-88a7fcb0f917 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:05.761451 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 354/694] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:29:05.762718 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9007f7-9593-411d-b56a-4ecdcc2b31fa None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:05.777387 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 341/695] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:29:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:05.782893 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:05.783037 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a89630-2650-465d-b1fe-88a7fcb0f917 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:05.907344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9007f7-9593-411d-b56a-4ecdcc2b31fa None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:05.920467 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a89630-2650-465d-b1fe-88a7fcb0f917 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:05.940366 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:05.952505 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16afff90-0bad-42f8-a615-0d5ceee21613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:05.953598 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16afff90-0bad-42f8-a615-0d5ceee21613 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:05.954020 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16afff90-0bad-42f8-a615-0d5ceee21613 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:05.954403 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16afff90-0bad-42f8-a615-0d5ceee21613 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:05.963254 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16afff90-0bad-42f8-a615-0d5ceee21613 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:05.965138 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16afff90-0bad-42f8-a615-0d5ceee21613 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:05.972434 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca517088-e88b-4e74-83ee-240d9fa77e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:05.973470 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca517088-e88b-4e74-83ee-240d9fa77e5f None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:05.973863 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca517088-e88b-4e74-83ee-240d9fa77e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:05.974246 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca517088-e88b-4e74-83ee-240d9fa77e5f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:05.982333 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-16afff90-0bad-42f8-a615-0d5ceee21613 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:29:05.982333 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:05.982333 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:05.982333 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:05.982333 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:05.982333 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:05.982333 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:05.982333 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:05.982333 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:05.982333 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:05.982333 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:05.982333 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:05.982333 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 16:29:05.982333 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 16:29:05.982333 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 16:29:05.983299 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 16:29:05.983299 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:05.983299 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:05.983299 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:05.983299 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:05.983299 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:05.983299 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:05.983299 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:05.983299 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:05.983299 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:05.983299 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:05.983299 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:05.983299 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:05.983299 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:05.983299 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:05.983299 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:05.983299 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:05.984208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 16:29:05.984208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 16:29:05.984208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 16:29:05.984208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 16:29:05.984208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 16:29:05.984208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 16:29:05.984208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:29:05.984208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:05.984208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 355/696] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:29:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:05.985784 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16afff90-0bad-42f8-a615-0d5ceee21613 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:05.986803 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca517088-e88b-4e74-83ee-240d9fa77e5f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:05.988664 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca517088-e88b-4e74-83ee-240d9fa77e5f None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:06.006007 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-ca517088-e88b-4e74-83ee-240d9fa77e5f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 16:29:06.006007 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:06.006007 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:06.006007 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:06.006007 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:06.006007 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:06.006007 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:06.006007 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:06.006007 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:06.006007 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:06.006007 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:06.006007 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:06.006007 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 16:29:06.006007 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 16:29:06.006007 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 16:29:06.006007 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 16:29:06.007019 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:06.007019 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:06.007019 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 16:29:06.007019 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:29:06.007019 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 16:29:06.007019 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:29:06.007019 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:06.007019 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:06.007019 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:06.007019 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:06.007019 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:06.007019 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:06.007019 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:06.007019 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:06.007019 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:06.007019 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:06.007019 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:06.007019 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:06.008017 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:06.008017 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:06.008017 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 16:29:06.008017 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 16:29:06.008017 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 16:29:06.008017 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 16:29:06.008017 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 16:29:06.008017 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 16:29:06.008017 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 16:29:06.008017 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:06.010957 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 342/697] 10.222.0.36 () {56 vars in 1093 bytes} [Thu May 19 16:29:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 16:29:06.012109 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca517088-e88b-4e74-83ee-240d9fa77e5f None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:06.072236 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:06.099564 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16afff90-0bad-42f8-a615-0d5ceee21613 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:06.154672 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca517088-e88b-4e74-83ee-240d9fa77e5f None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:06.156083 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f9b015-51cf-4516-8a14-b1966f82efcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:06.157733 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f9b015-51cf-4516-8a14-b1966f82efcb None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:06.158437 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f9b015-51cf-4516-8a14-b1966f82efcb None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:06.158941 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f9b015-51cf-4516-8a14-b1966f82efcb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:06.173766 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f9b015-51cf-4516-8a14-b1966f82efcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:06.175605 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f9b015-51cf-4516-8a14-b1966f82efcb None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:06.196887 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-b9f9b015-51cf-4516-8a14-b1966f82efcb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:06.196887 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:06.196887 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:06.196887 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:06.196887 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:06.196887 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:06.196887 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:06.196887 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:06.196887 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:06.196887 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:06.196887 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:06.196887 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:06.196887 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:29:06.196887 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:29:06.196887 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:29:06.197918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:29:06.197918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:06.197918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:06.197918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:06.197918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:06.197918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:06.197918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:06.197918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:06.197918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:06.197918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:06.197918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:06.197918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:06.197918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:06.197918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:06.197918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:06.197918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:06.197918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:06.198853 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:29:06.198853 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:29:06.198853 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:06.198853 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:06.201215 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 356/698] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:29:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:06.202336 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f9b015-51cf-4516-8a14-b1966f82efcb None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:06.211093 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558e02cb-f41c-40f4-848c-97d13602d92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:06.211793 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e02cb-f41c-40f4-848c-97d13602d92f None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:06.212079 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e02cb-f41c-40f4-848c-97d13602d92f None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:06.212359 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e02cb-f41c-40f4-848c-97d13602d92f None admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:06.213191 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558e02cb-f41c-40f4-848c-97d13602d92f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:06.213909 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558e02cb-f41c-40f4-848c-97d13602d92f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:06.215150 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558e02cb-f41c-40f4-848c-97d13602d92f None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:06.235609 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 343/699] 10.222.0.36 () {56 vars in 1148 bytes} [Thu May 19 16:29:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:06.236473 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558e02cb-f41c-40f4-848c-97d13602d92f None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:06.280427 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:06.317735 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f9b015-51cf-4516-8a14-b1966f82efcb None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:06.346863 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:06.357943 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a901ea56-7c4e-4dee-a75a-6da2f805d7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:06.358995 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901ea56-7c4e-4dee-a75a-6da2f805d7f0 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:06.359389 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901ea56-7c4e-4dee-a75a-6da2f805d7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:06.359807 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901ea56-7c4e-4dee-a75a-6da2f805d7f0 None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:06.360878 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a901ea56-7c4e-4dee-a75a-6da2f805d7f0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:06.361221 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a901ea56-7c4e-4dee-a75a-6da2f805d7f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:06.362946 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a901ea56-7c4e-4dee-a75a-6da2f805d7f0 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:06.370620 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558e02cb-f41c-40f4-848c-97d13602d92f None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:06.387733 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 357/700] 10.222.0.36 () {56 vars in 1133 bytes} [Thu May 19 16:29:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:06.388816 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a901ea56-7c4e-4dee-a75a-6da2f805d7f0 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:06.418602 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ffb06e-79b3-4ff4-be59-53739f444be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:06.419687 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ffb06e-79b3-4ff4-be59-53739f444be7 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:06.420124 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ffb06e-79b3-4ff4-be59-53739f444be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:06.420633 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ffb06e-79b3-4ff4-be59-53739f444be7 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:06.421627 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ffb06e-79b3-4ff4-be59-53739f444be7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:06.422454 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ffb06e-79b3-4ff4-be59-53739f444be7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:06.428438 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ffb06e-79b3-4ff4-be59-53739f444be7 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:06.443304 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 344/701] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:29:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:06.444732 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ffb06e-79b3-4ff4-be59-53739f444be7 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:06.502073 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a901ea56-7c4e-4dee-a75a-6da2f805d7f0 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:06.540783 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb5ae0-534c-4620-9557-307372e7d9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:06.541580 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb5ae0-534c-4620-9557-307372e7d9d0 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:06.541941 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb5ae0-534c-4620-9557-307372e7d9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:06.542332 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb5ae0-534c-4620-9557-307372e7d9d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:06.543293 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afb5ae0-534c-4620-9557-307372e7d9d0 None admin] RBAC: Adding query filter params (role.id=727026b648e1429ca3697e04800b328a, scope.project.id=631c76794a124d6da22c626e590e4352, user.id=ffe7cebc6f3b40d8a8e1866ed2da9822) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:06.543709 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afb5ae0-534c-4620-9557-307372e7d9d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:06.545294 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afb5ae0-534c-4620-9557-307372e7d9d0 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:06.552322 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 358/702] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:29:06 2022] GET /identity/v3/role_assignments?user.id=ffe7cebc6f3b40d8a8e1866ed2da9822&scope.project.id=631c76794a124d6da22c626e590e4352&role.id=727026b648e1429ca3697e04800b328a => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:06.553429 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb5ae0-534c-4620-9557-307372e7d9d0 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:06.591752 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:06.612503 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ffb06e-79b3-4ff4-be59-53739f444be7 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:06.617787 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:06.686424 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740afd1e-2572-4ce4-b4e9-eaca4d78231b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:06.687130 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740afd1e-2572-4ce4-b4e9-eaca4d78231b None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:06.687521 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740afd1e-2572-4ce4-b4e9-eaca4d78231b None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:06.687908 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740afd1e-2572-4ce4-b4e9-eaca4d78231b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:06.720672 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb5ae0-534c-4620-9557-307372e7d9d0 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:06.798519 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728da2a3-0090-40d5-98bf-3df10c38481f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:06.799711 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728da2a3-0090-40d5-98bf-3df10c38481f None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:06.801123 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728da2a3-0090-40d5-98bf-3df10c38481f None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:06.801858 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728da2a3-0090-40d5-98bf-3df10c38481f None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:06.804033 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728da2a3-0090-40d5-98bf-3df10c38481f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:06.805963 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728da2a3-0090-40d5-98bf-3df10c38481f None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:06.833532 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 359/703] 10.222.0.36 () {60 vars in 1132 bytes} [Thu May 19 16:29:06 2022] POST /identity/v3/users => generated 254 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:29:06.833911 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740afd1e-2572-4ce4-b4e9-eaca4d78231b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:06.835563 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740afd1e-2572-4ce4-b4e9-eaca4d78231b None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:06.954548 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:06.970060 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 345/704] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:29:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 16:29:06.975949 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728da2a3-0090-40d5-98bf-3df10c38481f None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:07.121343 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728da2a3-0090-40d5-98bf-3df10c38481f None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:07.130054 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:07.170669 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2976d0b2-a40e-49fa-a39b-dd59dd2ed5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:07.171499 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2976d0b2-a40e-49fa-a39b-dd59dd2ed5f4 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:07.171780 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2976d0b2-a40e-49fa-a39b-dd59dd2ed5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:07.172056 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2976d0b2-a40e-49fa-a39b-dd59dd2ed5f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:07.185537 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2976d0b2-a40e-49fa-a39b-dd59dd2ed5f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:07.187005 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2976d0b2-a40e-49fa-a39b-dd59dd2ed5f4 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:07.206451 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-2976d0b2-a40e-49fa-a39b-dd59dd2ed5f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:07.206451 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:07.206451 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:07.206451 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:07.206451 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:07.206451 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:07.206451 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:07.206451 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:07.206451 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:07.206451 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:07.206451 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:07.206451 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:07.206451 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:29:07.206451 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:29:07.206451 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:29:07.207371 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:29:07.207371 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:07.207371 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:07.207371 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:07.207371 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:07.207371 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:07.207371 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:07.207371 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:07.207371 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:07.207371 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:07.207371 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:07.207371 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:07.207371 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:07.207371 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:07.207371 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:07.207371 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:07.207371 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:07.208931 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:29:07.208931 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:29:07.208931 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:07.208931 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:07.208931 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 360/705] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:29:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:07.213222 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740afd1e-2572-4ce4-b4e9-eaca4d78231b None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:07.222093 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ed238-3e61-46f8-a8c9-944796d48535 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:07.222380 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ed238-3e61-46f8-a8c9-944796d48535 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:07.225197 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ed238-3e61-46f8-a8c9-944796d48535 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:07.225197 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 361/706] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:29:07.230992 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3a383-f252-41dc-8a71-f582ad5f6787 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:07.230992 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3a383-f252-41dc-8a71-f582ad5f6787 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:07.231258 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3a383-f252-41dc-8a71-f582ad5f6787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:07.251280 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-c1e3a383-f252-41dc-8a71-f582ad5f6787 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:07.252502 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-c1e3a383-f252-41dc-8a71-f582ad5f6787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:07.364353 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740afd1e-2572-4ce4-b4e9-eaca4d78231b None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:07.378791 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 362/707] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 16:29:07.402599 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f275f9e-9caf-4474-a205-c98f2bf8fda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:07.403283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f275f9e-9caf-4474-a205-c98f2bf8fda0 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:07.403572 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f275f9e-9caf-4474-a205-c98f2bf8fda0 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:07.403852 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f275f9e-9caf-4474-a205-c98f2bf8fda0 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:07.404651 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f275f9e-9caf-4474-a205-c98f2bf8fda0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:07.404966 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f275f9e-9caf-4474-a205-c98f2bf8fda0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:07.406440 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f275f9e-9caf-4474-a205-c98f2bf8fda0 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:07.414221 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e3a383-f252-41dc-8a71-f582ad5f6787 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:07.420105 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 346/708] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:29:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:07.426107 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f275f9e-9caf-4474-a205-c98f2bf8fda0 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:07.485821 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:07.548293 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f275f9e-9caf-4474-a205-c98f2bf8fda0 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:07.611470 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ef3564-959a-4683-903b-597152f093eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:07.612623 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef3564-959a-4683-903b-597152f093eb None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:07.612911 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef3564-959a-4683-903b-597152f093eb None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:07.613204 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef3564-959a-4683-903b-597152f093eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:07.625304 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ef3564-959a-4683-903b-597152f093eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:07.626845 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ef3564-959a-4683-903b-597152f093eb None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:07.637571 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:07.645606 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e3a383-f252-41dc-8a71-f582ad5f6787 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:07.665730 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-99ef3564-959a-4683-903b-597152f093eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:29:07.665730 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:07.665730 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:07.665730 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:07.665730 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:07.665730 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:07.665730 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:07.665730 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:07.665730 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:07.665730 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:07.665730 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:07.665730 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:07.665730 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 16:29:07.665730 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 16:29:07.665730 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 16:29:07.666662 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 16:29:07.666662 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:07.666662 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:07.666662 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:07.666662 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:07.666662 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:07.666662 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:07.666662 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:07.666662 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:07.666662 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:07.666662 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:07.666662 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:07.666662 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:07.666662 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:07.666662 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:07.666662 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:07.666662 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:07.667545 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 16:29:07.667545 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 16:29:07.667545 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 16:29:07.667545 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 16:29:07.667545 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 16:29:07.667545 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 16:29:07.667545 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:29:07.667545 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:07.667545 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 347/709] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:29:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 16:29:07.673555 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ef3564-959a-4683-903b-597152f093eb None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:07.688979 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed03555-6037-44b8-8f39-bd82096a264b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:07.689936 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed03555-6037-44b8-8f39-bd82096a264b None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:07.690227 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed03555-6037-44b8-8f39-bd82096a264b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:07.690504 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed03555-6037-44b8-8f39-bd82096a264b None admin] PATH_INFO: `/v3/services/network` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:07.691185 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed03555-6037-44b8-8f39-bd82096a264b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:07.692603 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed03555-6037-44b8-8f39-bd82096a264b None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:07.710060 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-2ed03555-6037-44b8-8f39-bd82096a264b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 16:29:07.710060 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:07.710060 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 16:29:07.710060 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 16:29:07.710060 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 16:29:07.710060 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 16:29:07.710060 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 16:29:07.710060 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:29:07.710060 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 16:29:07.710060 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:07.710060 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 16:29:07.710060 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:07.710060 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:07.710060 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:07.710060 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:07.710060 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:07.710060 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:07.711160 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:07.711160 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:07.711160 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:07.711160 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:07.711160 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:07.711160 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:07.711160 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 16:29:07.711160 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 16:29:07.711160 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 16:29:07.711160 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 16:29:07.711160 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:07.711160 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:07.711160 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:07.711160 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:07.711160 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:07.711160 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:07.711989 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:07.711989 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:07.711989 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:07.711989 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:07.711989 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:07.711989 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:07.711989 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:07.711989 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:07.711989 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:07.711989 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:07.711989 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 16:29:07.711989 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:29:07.711989 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 16:29:07.711989 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:07.711989 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 363/710] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:29:07 2022] GET /identity/v3/services/network => generated 88 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:07.717361 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed03555-6037-44b8-8f39-bd82096a264b None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:07.842299 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed03555-6037-44b8-8f39-bd82096a264b None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:07.895722 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fa3401-31a6-4cae-8d5e-a8c49938e9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:07.897312 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa3401-31a6-4cae-8d5e-a8c49938e9a8 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:07.897932 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa3401-31a6-4cae-8d5e-a8c49938e9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:07.898540 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa3401-31a6-4cae-8d5e-a8c49938e9a8 None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:07.900638 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fa3401-31a6-4cae-8d5e-a8c49938e9a8 None admin] RBAC: Adding query filter params (name=network) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:07.901637 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fa3401-31a6-4cae-8d5e-a8c49938e9a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:07.904087 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fa3401-31a6-4cae-8d5e-a8c49938e9a8 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:07.911343 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 364/711] 10.222.0.36 () {56 vars in 1112 bytes} [Thu May 19 16:29:07 2022] GET /identity/v3/services?name=network => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:07.913057 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea75d5c-c2c0-4e21-96d2-b5a7f6792f88 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:07.913507 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea75d5c-c2c0-4e21-96d2-b5a7f6792f88 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:07.913914 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea75d5c-c2c0-4e21-96d2-b5a7f6792f88 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:07.916035 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 365/712] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:29:07.925177 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb3155-4f62-4b89-8abd-8ba9f598f893 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:07.925177 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb3155-4f62-4b89-8abd-8ba9f598f893 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:07.925177 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb3155-4f62-4b89-8abd-8ba9f598f893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:07.945353 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-a6bb3155-4f62-4b89-8abd-8ba9f598f893 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:07.946874 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-a6bb3155-4f62-4b89-8abd-8ba9f598f893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:07.959930 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:07.976628 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ef3564-959a-4683-903b-597152f093eb None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:07.990565 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:08.020717 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf3c2f6-2e24-4bd3-91c5-614ee40ba508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:08.022068 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf3c2f6-2e24-4bd3-91c5-614ee40ba508 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:08.022816 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf3c2f6-2e24-4bd3-91c5-614ee40ba508 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:08.023243 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf3c2f6-2e24-4bd3-91c5-614ee40ba508 None admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:08.024356 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf3c2f6-2e24-4bd3-91c5-614ee40ba508 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:08.024829 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf3c2f6-2e24-4bd3-91c5-614ee40ba508 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:08.027014 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf3c2f6-2e24-4bd3-91c5-614ee40ba508 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:08.034491 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 366/713] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 16:29:08.035664 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb3155-4f62-4b89-8abd-8ba9f598f893 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:08.044014 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 348/714] 10.222.0.36 () {56 vars in 1148 bytes} [Thu May 19 16:29:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:29:08.050247 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf3c2f6-2e24-4bd3-91c5-614ee40ba508 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:08.167984 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf3c2f6-2e24-4bd3-91c5-614ee40ba508 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:08.175449 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb3155-4f62-4b89-8abd-8ba9f598f893 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:08.209013 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b73f37c-aaeb-4883-a6eb-b2a73648ca46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:08.210017 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73f37c-aaeb-4883-a6eb-b2a73648ca46 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:08.210437 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73f37c-aaeb-4883-a6eb-b2a73648ca46 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:08.210851 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73f37c-aaeb-4883-a6eb-b2a73648ca46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:08.212154 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b73f37c-aaeb-4883-a6eb-b2a73648ca46 None admin] RBAC: Adding query filter params (role.id=e2915fe109244a07a9be96bd2b219fdd, scope.project.id=631c76794a124d6da22c626e590e4352, user.id=58bdeada00b74b7b8e186e6dc51e6dfb) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:08.212631 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b73f37c-aaeb-4883-a6eb-b2a73648ca46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:08.213722 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc194fb-9f7b-45a3-98f0-bc9734fff0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:08.214610 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc194fb-9f7b-45a3-98f0-bc9734fff0d2 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:08.214920 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b73f37c-aaeb-4883-a6eb-b2a73648ca46 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:08.215651 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc194fb-9f7b-45a3-98f0-bc9734fff0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:08.216122 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc194fb-9f7b-45a3-98f0-bc9734fff0d2 None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:08.217032 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc194fb-9f7b-45a3-98f0-bc9734fff0d2 None admin] RBAC: Adding query filter params (type=network) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:08.217568 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc194fb-9f7b-45a3-98f0-bc9734fff0d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:08.219206 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc194fb-9f7b-45a3-98f0-bc9734fff0d2 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:08.226290 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 367/715] 10.222.0.36 () {56 vars in 1112 bytes} [Thu May 19 16:29:08 2022] GET /identity/v3/services?type=network => generated 124 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:08.227432 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc194fb-9f7b-45a3-98f0-bc9734fff0d2 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:08.233301 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 349/716] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:29:08 2022] GET /identity/v3/role_assignments?user.id=58bdeada00b74b7b8e186e6dc51e6dfb&scope.project.id=631c76794a124d6da22c626e590e4352&role.id=e2915fe109244a07a9be96bd2b219fdd => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:08.258601 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:08.393913 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc194fb-9f7b-45a3-98f0-bc9734fff0d2 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:08.452076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc976aa-ca47-46ad-a0a0-13a7457bd16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:08.452837 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc976aa-ca47-46ad-a0a0-13a7457bd16f None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:08.453119 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc976aa-ca47-46ad-a0a0-13a7457bd16f None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:08.454050 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc976aa-ca47-46ad-a0a0-13a7457bd16f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:08.454747 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc976aa-ca47-46ad-a0a0-13a7457bd16f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:08.456284 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc976aa-ca47-46ad-a0a0-13a7457bd16f None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:08.471275 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-dfc976aa-ca47-46ad-a0a0-13a7457bd16f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 16:29:08.471275 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:08.471275 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 16:29:08.471275 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 16:29:08.471275 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 16:29:08.471275 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 16:29:08.471275 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 16:29:08.471275 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:29:08.471275 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 16:29:08.471275 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:08.471275 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 16:29:08.471275 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:08.471275 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:08.471275 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:08.471275 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:08.471275 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:08.472732 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:08.472732 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:08.472732 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:08.472732 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:08.472732 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:08.472732 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:08.472732 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:08.472732 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 16:29:08.472732 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 16:29:08.472732 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 16:29:08.472732 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 16:29:08.472732 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:08.472732 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:08.472732 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:08.472732 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:08.472732 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:08.473760 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:08.473760 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:08.473760 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:08.473760 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:08.473760 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:08.473760 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:08.473760 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:08.473760 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:08.473760 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:08.473760 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:08.473760 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:08.473760 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 16:29:08.473760 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:29:08.473760 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 16:29:08.473760 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:08.473760 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 368/717] 10.222.0.36 () {56 vars in 1129 bytes} [Thu May 19 16:29:08 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:08.477406 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b73f37c-aaeb-4883-a6eb-b2a73648ca46 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:08.518003 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:08.608397 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b73f37c-aaeb-4883-a6eb-b2a73648ca46 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:08.653627 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea72714-7bff-4848-ae5f-56570a41ed33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:08.654360 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea72714-7bff-4848-ae5f-56570a41ed33 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:08.654653 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea72714-7bff-4848-ae5f-56570a41ed33 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:08.654931 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea72714-7bff-4848-ae5f-56570a41ed33 None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:08.655659 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea72714-7bff-4848-ae5f-56570a41ed33 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:08.655971 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea72714-7bff-4848-ae5f-56570a41ed33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:08.657406 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea72714-7bff-4848-ae5f-56570a41ed33 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:08.665382 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 350/718] 10.222.0.36 () {56 vars in 1124 bytes} [Thu May 19 16:29:08 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:08.670545 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc976aa-ca47-46ad-a0a0-13a7457bd16f None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:08.762199 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:08.790889 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc976aa-ca47-46ad-a0a0-13a7457bd16f None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:08.832531 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f09dbb-a935-4e82-a5e7-725dbcb5c80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:08.833627 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f09dbb-a935-4e82-a5e7-725dbcb5c80b None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:08.834124 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f09dbb-a935-4e82-a5e7-725dbcb5c80b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:08.834603 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f09dbb-a935-4e82-a5e7-725dbcb5c80b None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:08.835532 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f09dbb-a935-4e82-a5e7-725dbcb5c80b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:08.836062 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f09dbb-a935-4e82-a5e7-725dbcb5c80b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:08.837700 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f09dbb-a935-4e82-a5e7-725dbcb5c80b None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:08.849267 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 369/719] 10.222.0.36 () {56 vars in 1124 bytes} [Thu May 19 16:29:08 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:09.677103 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947af5e8-314e-40b3-ab46-5fa470ef8173 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:09.677658 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947af5e8-314e-40b3-ab46-5fa470ef8173 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:09.677776 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947af5e8-314e-40b3-ab46-5fa470ef8173 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:09.679344 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 351/720] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:29:09.684992 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ff587-b1b1-48dd-a02e-97760e331b9a None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:09.685913 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ff587-b1b1-48dd-a02e-97760e331b9a None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:09.686414 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ff587-b1b1-48dd-a02e-97760e331b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:09.705471 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-792ff587-b1b1-48dd-a02e-97760e331b9a None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:09.707021 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-792ff587-b1b1-48dd-a02e-97760e331b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:09.805616 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 370/721] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:29:09.844585 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947af5e8-314e-40b3-ab46-5fa470ef8173 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:09.855290 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8acd01-bbaa-4330-8720-4644e516379d None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:09.857847 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8acd01-bbaa-4330-8720-4644e516379d None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:09.858348 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8acd01-bbaa-4330-8720-4644e516379d None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:09.861966 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 371/722] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:09 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:29:09.867298 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d511f-28c8-4866-8420-49d107153ff8 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:09.867894 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d511f-28c8-4866-8420-49d107153ff8 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:09.868399 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d511f-28c8-4866-8420-49d107153ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:09.897540 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-6d7d511f-28c8-4866-8420-49d107153ff8 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:09.898745 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-6d7d511f-28c8-4866-8420-49d107153ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:10.004364 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:10.017281 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 372/723] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 16:29:10.039067 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7d511f-28c8-4866-8420-49d107153ff8 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:10.043538 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:10.057941 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947af5e8-314e-40b3-ab46-5fa470ef8173 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:10.134109 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da02ac0-9ef9-49e9-b4c9-acf8228cf297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:10.135018 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da02ac0-9ef9-49e9-b4c9-acf8228cf297 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:10.135314 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da02ac0-9ef9-49e9-b4c9-acf8228cf297 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:10.135587 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da02ac0-9ef9-49e9-b4c9-acf8228cf297 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:10.159052 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7d511f-28c8-4866-8420-49d107153ff8 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:10.161300 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da02ac0-9ef9-49e9-b4c9-acf8228cf297 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:29:10.177864 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-9da02ac0-9ef9-49e9-b4c9-acf8228cf297 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:29:10.177864 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:10.177864 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:10.177864 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:10.177864 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:10.177864 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:10.177864 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:10.177864 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:10.177864 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:10.177864 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:10.177864 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:10.177864 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:10.177864 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:29:10.177864 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:29:10.177864 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:29:10.177864 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:29:10.178901 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:29:10.178901 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:29:10.178901 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:29:10.178901 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:29:10.178901 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:10.178901 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:10.178901 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:10.178901 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:10.178901 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:10.178901 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:10.178901 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:10.178901 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:10.178901 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:10.178901 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:10.178901 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:10.178901 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:10.180195 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:10.180195 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:10.180195 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:10.180195 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:10.180195 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:29:10.180195 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:29:10.180195 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:29:10.180195 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:29:10.180195 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:29:10.180195 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:29:10.180195 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:29:10.180195 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:10.180195 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 352/724] 10.222.0.36 () {56 vars in 1096 bytes} [Thu May 19 16:29:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:10.194850 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da02ac0-9ef9-49e9-b4c9-acf8228cf297 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:10.214174 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db314e2-c4f3-4a16-8697-04a0509ec8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:10.214996 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db314e2-c4f3-4a16-8697-04a0509ec8bd None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:10.215278 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db314e2-c4f3-4a16-8697-04a0509ec8bd None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:10.215553 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db314e2-c4f3-4a16-8697-04a0509ec8bd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:10.241781 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db314e2-c4f3-4a16-8697-04a0509ec8bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:29:10.249823 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-9db314e2-c4f3-4a16-8697-04a0509ec8bd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 16:29:10.249823 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:10.249823 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:10.249823 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:10.249823 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:10.249823 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:10.249823 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:10.249823 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:10.249823 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:10.249823 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:10.249823 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:10.249823 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:10.249823 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:29:10.249823 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:29:10.249823 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:29:10.249823 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:29:10.252832 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:29:10.252832 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:29:10.252832 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:29:10.252832 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:29:10.252832 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:10.252832 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:10.252832 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:10.252832 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:10.252832 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:10.252832 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:10.252832 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:10.252832 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:10.252832 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:10.252832 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:10.252832 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:10.252832 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:10.254948 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:10.254948 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:10.254948 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:10.254948 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:10.254948 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:29:10.254948 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:29:10.254948 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:29:10.254948 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:29:10.254948 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:29:10.254948 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:29:10.254948 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 16:29:10.254948 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:10.254948 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 373/725] 10.222.0.36 () {56 vars in 1102 bytes} [Thu May 19 16:29:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:10.265402 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db314e2-c4f3-4a16-8697-04a0509ec8bd None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:10.374242 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da02ac0-9ef9-49e9-b4c9-acf8228cf297 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:10.429769 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:10.503639 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33945f07-4b35-40ac-a406-58e9edb90df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:10.505402 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33945f07-4b35-40ac-a406-58e9edb90df1 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:10.506185 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33945f07-4b35-40ac-a406-58e9edb90df1 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:10.506896 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33945f07-4b35-40ac-a406-58e9edb90df1 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:10.508640 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33945f07-4b35-40ac-a406-58e9edb90df1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:10.511784 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33945f07-4b35-40ac-a406-58e9edb90df1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:10.514624 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33945f07-4b35-40ac-a406-58e9edb90df1 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:10.553553 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 353/726] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:29:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:10.558206 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33945f07-4b35-40ac-a406-58e9edb90df1 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:10.565204 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db314e2-c4f3-4a16-8697-04a0509ec8bd None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:10.596992 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:10.613117 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953305b5-8325-4f0a-a6c1-1c8bca7e87d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:10.615109 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953305b5-8325-4f0a-a6c1-1c8bca7e87d5 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:10.615664 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953305b5-8325-4f0a-a6c1-1c8bca7e87d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:10.616205 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953305b5-8325-4f0a-a6c1-1c8bca7e87d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:10.617259 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953305b5-8325-4f0a-a6c1-1c8bca7e87d5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:10.617959 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953305b5-8325-4f0a-a6c1-1c8bca7e87d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:10.620001 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953305b5-8325-4f0a-a6c1-1c8bca7e87d5 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:10.630896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 374/727] 10.222.0.36 () {56 vars in 1103 bytes} [Thu May 19 16:29:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:10.640615 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953305b5-8325-4f0a-a6c1-1c8bca7e87d5 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:10.792259 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33945f07-4b35-40ac-a406-58e9edb90df1 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:10.855092 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953305b5-8325-4f0a-a6c1-1c8bca7e87d5 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:10.858694 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ed157-21cc-42c7-b6a3-43a9c5cf22ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:10.859848 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ed157-21cc-42c7-b6a3-43a9c5cf22ee None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:10.860148 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ed157-21cc-42c7-b6a3-43a9c5cf22ee None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:10.860433 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ed157-21cc-42c7-b6a3-43a9c5cf22ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:10.884082 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ed157-21cc-42c7-b6a3-43a9c5cf22ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:10.885603 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ed157-21cc-42c7-b6a3-43a9c5cf22ee None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:10.915535 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:10.919053 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-533ed157-21cc-42c7-b6a3-43a9c5cf22ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:10.919053 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:10.919053 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:10.919053 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:10.919053 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:10.919053 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:10.919053 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:10.919053 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:10.919053 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:10.919053 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:10.919053 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:10.919053 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:10.919053 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:29:10.919053 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:29:10.919053 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:29:10.920005 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:29:10.920005 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:10.920005 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:10.920005 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:10.920005 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:10.920005 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:10.920005 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:10.920005 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:10.920005 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:10.920005 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:10.920005 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:10.920005 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:10.920005 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:10.920005 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:10.920005 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:10.920005 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:10.920005 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:10.920896 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:29:10.920896 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:29:10.920896 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:10.920896 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:10.926299 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 354/728] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:29:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:10.932563 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ed157-21cc-42c7-b6a3-43a9c5cf22ee None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:10.963014 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5fc32-2aa1-4345-98ab-1fd299317206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:10.964323 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5fc32-2aa1-4345-98ab-1fd299317206 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:10.964908 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5fc32-2aa1-4345-98ab-1fd299317206 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:10.965455 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5fc32-2aa1-4345-98ab-1fd299317206 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:10.989057 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d5fc32-2aa1-4345-98ab-1fd299317206 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:10.990990 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d5fc32-2aa1-4345-98ab-1fd299317206 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:11.033711 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-35d5fc32-2aa1-4345-98ab-1fd299317206 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:11.033711 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:11.033711 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:11.033711 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:11.033711 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:11.033711 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:11.033711 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:11.033711 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:11.033711 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:11.033711 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:11.033711 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:11.033711 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:11.033711 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:29:11.033711 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:29:11.033711 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:29:11.034675 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:29:11.034675 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:11.034675 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:11.034675 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:11.034675 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:11.034675 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:11.034675 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:11.034675 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:11.034675 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:11.034675 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:11.034675 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:11.034675 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:11.034675 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:11.034675 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:11.034675 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:11.034675 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:11.034675 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:11.035541 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:29:11.035541 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:29:11.035541 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:11.035541 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:11.044536 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 375/729] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:29:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 409 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 16:29:11.048416 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5fc32-2aa1-4345-98ab-1fd299317206 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:11.091757 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:11.155826 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ed157-21cc-42c7-b6a3-43a9c5cf22ee None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:11.193731 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f2fe20-7459-4c90-b173-5fab4388c314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:11.194519 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f2fe20-7459-4c90-b173-5fab4388c314 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:11.194809 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f2fe20-7459-4c90-b173-5fab4388c314 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:11.195085 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f2fe20-7459-4c90-b173-5fab4388c314 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:11.195948 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f2fe20-7459-4c90-b173-5fab4388c314 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:11.196361 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f2fe20-7459-4c90-b173-5fab4388c314 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:11.197798 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f2fe20-7459-4c90-b173-5fab4388c314 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:11.219619 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 355/730] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:29:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:29:11.221287 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8131cd5-a097-48f4-9096-6e6de6a01255 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:11.221581 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8131cd5-a097-48f4-9096-6e6de6a01255 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:11.221861 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8131cd5-a097-48f4-9096-6e6de6a01255 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:11.224174 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 356/731] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:29:11.225055 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8131cd5-a097-48f4-9096-6e6de6a01255 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:11.245091 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5fc32-2aa1-4345-98ab-1fd299317206 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:11.284592 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85efdf6c-a097-42c6-8021-5540bd085fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:11.285307 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85efdf6c-a097-42c6-8021-5540bd085fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:11.285601 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85efdf6c-a097-42c6-8021-5540bd085fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:11.285885 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85efdf6c-a097-42c6-8021-5540bd085fc0 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:11.286655 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85efdf6c-a097-42c6-8021-5540bd085fc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:11.286981 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85efdf6c-a097-42c6-8021-5540bd085fc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:11.288375 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85efdf6c-a097-42c6-8021-5540bd085fc0 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:11.301306 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 376/732] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:29:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:11.302571 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02b66b-68ea-4955-9524-113d18706980 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:11.302946 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02b66b-68ea-4955-9524-113d18706980 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:11.303232 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02b66b-68ea-4955-9524-113d18706980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:11.321772 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-6a02b66b-68ea-4955-9524-113d18706980 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:11.323718 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-6a02b66b-68ea-4955-9524-113d18706980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:11.355569 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:11.362234 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8131cd5-a097-48f4-9096-6e6de6a01255 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:11.399788 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:11.405677 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ecb6c6-0913-4a33-a934-6b9ccbec5a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:11.406532 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ecb6c6-0913-4a33-a934-6b9ccbec5a11 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:11.411398 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ecb6c6-0913-4a33-a934-6b9ccbec5a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:11.411543 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 377/733] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:29:11.411639 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ecb6c6-0913-4a33-a934-6b9ccbec5a11 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:11.411726 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c4f0b-d283-4604-958d-e63c909e9bba None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:11.411726 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c4f0b-d283-4604-958d-e63c909e9bba None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:11.411726 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c4f0b-d283-4604-958d-e63c909e9bba None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:11.411726 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 378/734] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:29:11.411726 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c4f0b-d283-4604-958d-e63c909e9bba None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:11.428120 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ecb6c6-0913-4a33-a934-6b9ccbec5a11 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:11.429638 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ecb6c6-0913-4a33-a934-6b9ccbec5a11 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:11.445664 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-d3ecb6c6-0913-4a33-a934-6b9ccbec5a11 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 16:29:11.445664 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:11.445664 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:11.445664 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:11.445664 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:11.445664 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:11.445664 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:11.445664 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:11.445664 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:11.445664 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:11.445664 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:11.445664 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:11.445664 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 16:29:11.445664 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 16:29:11.445664 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 16:29:11.445664 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 16:29:11.446691 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:11.446691 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:11.446691 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 16:29:11.446691 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:29:11.446691 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 16:29:11.446691 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:29:11.446691 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:11.446691 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:11.446691 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:11.446691 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:11.446691 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:11.446691 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:11.446691 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:11.446691 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:11.446691 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:11.446691 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:11.446691 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:11.446691 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:11.447644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:11.447644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:11.447644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 16:29:11.447644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 16:29:11.447644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 16:29:11.447644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 16:29:11.447644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 16:29:11.447644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 16:29:11.447644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 16:29:11.447644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:11.447644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 357/735] 10.222.0.36 () {56 vars in 1096 bytes} [Thu May 19 16:29:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:11.448760 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7e122-d7d4-4f45-aa48-38557ed5e023 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:11.448760 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7e122-d7d4-4f45-aa48-38557ed5e023 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:11.448760 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7e122-d7d4-4f45-aa48-38557ed5e023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:11.465859 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-a8b7e122-d7d4-4f45-aa48-38557ed5e023 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:11.467048 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-a8b7e122-d7d4-4f45-aa48-38557ed5e023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:11.538580 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c4f0b-d283-4604-958d-e63c909e9bba None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:11.547943 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 358/736] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:29:11.549201 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b7e122-d7d4-4f45-aa48-38557ed5e023 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:11.574779 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4312212d-7de9-4442-bb7f-d78eb58e1344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:11.575513 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4312212d-7de9-4442-bb7f-d78eb58e1344 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:11.575796 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4312212d-7de9-4442-bb7f-d78eb58e1344 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:11.576071 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4312212d-7de9-4442-bb7f-d78eb58e1344 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:11.583472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4312212d-7de9-4442-bb7f-d78eb58e1344 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:11.584928 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4312212d-7de9-4442-bb7f-d78eb58e1344 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:11.600565 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-4312212d-7de9-4442-bb7f-d78eb58e1344 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 16:29:11.600565 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:11.600565 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:11.600565 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:11.600565 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:11.600565 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:11.600565 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:11.600565 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:11.600565 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:11.600565 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:11.600565 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:11.600565 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:11.600565 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 16:29:11.600565 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 16:29:11.600565 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 16:29:11.600565 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 16:29:11.601848 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:11.601848 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:11.601848 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 16:29:11.601848 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:29:11.601848 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 16:29:11.601848 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:29:11.601848 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:11.601848 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:11.601848 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:11.601848 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:11.601848 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:11.601848 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:11.601848 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:11.601848 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:11.601848 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:11.601848 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:11.601848 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:11.601848 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:11.603250 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:11.603250 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:11.603250 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 16:29:11.603250 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 16:29:11.603250 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 16:29:11.603250 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 16:29:11.603250 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 16:29:11.603250 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 16:29:11.603250 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 16:29:11.603250 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:11.605300 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 379/737] 10.222.0.36 () {56 vars in 1117 bytes} [Thu May 19 16:29:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:11.606955 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4312212d-7de9-4442-bb7f-d78eb58e1344 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:11.635959 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:11.675088 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b7e122-d7d4-4f45-aa48-38557ed5e023 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:11.700003 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:11.712500 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be15e2d9-84cb-4baf-9da8-11a26f2f56c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:11.713244 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be15e2d9-84cb-4baf-9da8-11a26f2f56c8 None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:11.713530 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be15e2d9-84cb-4baf-9da8-11a26f2f56c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:11.713808 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be15e2d9-84cb-4baf-9da8-11a26f2f56c8 None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:11.714545 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be15e2d9-84cb-4baf-9da8-11a26f2f56c8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:11.715918 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be15e2d9-84cb-4baf-9da8-11a26f2f56c8 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:11.730150 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 359/738] 10.222.0.36 () {60 vars in 1141 bytes} [Thu May 19 16:29:11 2022] POST /identity/v3/services => generated 240 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:29:11.733976 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4312212d-7de9-4442-bb7f-d78eb58e1344 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:11.734144 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be15e2d9-84cb-4baf-9da8-11a26f2f56c8 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:11.786381 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb577bd0-df4a-42c2-b985-b3f04aa85b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:11.787085 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb577bd0-df4a-42c2-b985-b3f04aa85b70 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:11.787404 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb577bd0-df4a-42c2-b985-b3f04aa85b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:11.787689 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb577bd0-df4a-42c2-b985-b3f04aa85b70 None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:11.788571 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb577bd0-df4a-42c2-b985-b3f04aa85b70 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:11.788891 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb577bd0-df4a-42c2-b985-b3f04aa85b70 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:11.790493 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb577bd0-df4a-42c2-b985-b3f04aa85b70 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:11.813878 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 380/739] 10.222.0.36 () {56 vars in 1135 bytes} [Thu May 19 16:29:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:11.814774 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb577bd0-df4a-42c2-b985-b3f04aa85b70 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:11.877349 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be15e2d9-84cb-4baf-9da8-11a26f2f56c8 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:11.904942 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:11.926677 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a208d-5fa2-4f05-a936-816f0d96aa99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:11.927503 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a208d-5fa2-4f05-a936-816f0d96aa99 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:11.927883 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a208d-5fa2-4f05-a936-816f0d96aa99 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:11.928077 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a208d-5fa2-4f05-a936-816f0d96aa99 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:11.938068 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb577bd0-df4a-42c2-b985-b3f04aa85b70 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:11.948345 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a208d-5fa2-4f05-a936-816f0d96aa99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:29:11.963498 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-043a208d-5fa2-4f05-a936-816f0d96aa99 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:29:11.963498 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:11.963498 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:11.963498 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:11.963498 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:11.963498 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:11.963498 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:11.963498 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:11.963498 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:11.963498 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:11.963498 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:11.963498 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:11.963498 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:29:11.963498 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:29:11.963498 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:29:11.963498 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:29:11.964473 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:29:11.964473 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:29:11.964473 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:29:11.964473 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:29:11.964473 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:11.964473 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:11.964473 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:11.964473 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:11.964473 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:11.964473 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:11.964473 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:11.964473 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:11.964473 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:11.964473 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:11.964473 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:11.964473 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:11.965405 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:11.965405 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:11.965405 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:11.965405 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:11.965405 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:29:11.965405 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:29:11.965405 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:29:11.965405 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:29:11.965405 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:29:11.965405 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:29:11.965405 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:29:11.965405 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:11.966046 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 360/740] 10.222.0.36 () {56 vars in 1096 bytes} [Thu May 19 16:29:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:11.968167 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff9990a-cc49-492c-8472-f295706d2293 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:11.968688 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff9990a-cc49-492c-8472-f295706d2293 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:11.969778 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff9990a-cc49-492c-8472-f295706d2293 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:11.972360 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 361/741] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:29:11.973623 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff9990a-cc49-492c-8472-f295706d2293 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:11.985729 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f530b-9420-4079-90ac-4d9b165472a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:11.986420 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f530b-9420-4079-90ac-4d9b165472a3 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:11.986705 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f530b-9420-4079-90ac-4d9b165472a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:11.986980 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f530b-9420-4079-90ac-4d9b165472a3 None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:11.987826 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94f530b-9420-4079-90ac-4d9b165472a3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:11.988134 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94f530b-9420-4079-90ac-4d9b165472a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:11.989607 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94f530b-9420-4079-90ac-4d9b165472a3 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:12.009225 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 381/742] 10.222.0.36 () {56 vars in 1149 bytes} [Thu May 19 16:29:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:12.010413 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f530b-9420-4079-90ac-4d9b165472a3 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:12.036175 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:12.099482 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff9990a-cc49-492c-8472-f295706d2293 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:12.122698 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f530b-9420-4079-90ac-4d9b165472a3 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:12.141119 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ac8f5-2149-4213-b345-bab60a333f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:12.141757 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ac8f5-2149-4213-b345-bab60a333f0f None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:12.142046 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ac8f5-2149-4213-b345-bab60a333f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:12.142421 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ac8f5-2149-4213-b345-bab60a333f0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:12.159258 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898a6ab-ff15-4386-b61a-625b9c9a4444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:12.159942 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898a6ab-ff15-4386-b61a-625b9c9a4444 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:12.160230 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898a6ab-ff15-4386-b61a-625b9c9a4444 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:12.160518 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898a6ab-ff15-4386-b61a-625b9c9a4444 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:12.161317 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6898a6ab-ff15-4386-b61a-625b9c9a4444 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:12.161633 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6898a6ab-ff15-4386-b61a-625b9c9a4444 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:12.163035 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6898a6ab-ff15-4386-b61a-625b9c9a4444 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:12.164653 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:12.173748 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 382/743] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:29:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:12.174983 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b343e-686c-4add-a6ee-c1889be5fcf9 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:12.175346 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b343e-686c-4add-a6ee-c1889be5fcf9 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:12.175694 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b343e-686c-4add-a6ee-c1889be5fcf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:12.192255 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-226b343e-686c-4add-a6ee-c1889be5fcf9 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:12.193597 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-226b343e-686c-4add-a6ee-c1889be5fcf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:12.226675 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332ac8f5-2149-4213-b345-bab60a333f0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:12.228276 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332ac8f5-2149-4213-b345-bab60a333f0f None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:12.265519 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 383/744] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:12 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 16:29:12.266972 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b343e-686c-4add-a6ee-c1889be5fcf9 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:12.276469 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:12.347449 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 362/745] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:29:11 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 16:29:12.347449 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ac8f5-2149-4213-b345-bab60a333f0f None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:12.384085 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b343e-686c-4add-a6ee-c1889be5fcf9 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:12.411175 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:12.420453 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8947ceea-ef56-4b62-a3ce-4a5cb34e1766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:12.421134 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8947ceea-ef56-4b62-a3ce-4a5cb34e1766 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:12.421458 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8947ceea-ef56-4b62-a3ce-4a5cb34e1766 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:12.421742 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8947ceea-ef56-4b62-a3ce-4a5cb34e1766 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:12.457529 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ac8f5-2149-4213-b345-bab60a333f0f None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:12.505820 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8947ceea-ef56-4b62-a3ce-4a5cb34e1766 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:12.507463 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8947ceea-ef56-4b62-a3ce-4a5cb34e1766 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:12.513769 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf17d18-1059-4440-9c2b-c07f2373134a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:12.514603 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf17d18-1059-4440-9c2b-c07f2373134a None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:12.514885 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf17d18-1059-4440-9c2b-c07f2373134a None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:12.515168 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf17d18-1059-4440-9c2b-c07f2373134a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:12.527381 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf17d18-1059-4440-9c2b-c07f2373134a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:12.529031 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf17d18-1059-4440-9c2b-c07f2373134a None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:12.532303 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:12.549856 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-cdf17d18-1059-4440-9c2b-c07f2373134a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:12.549856 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:12.549856 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:12.549856 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:12.549856 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:12.549856 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:12.549856 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:12.549856 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:12.549856 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:12.549856 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:12.549856 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:12.549856 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:12.549856 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:29:12.549856 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:29:12.549856 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:29:12.551647 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:29:12.551647 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:12.551647 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:12.551647 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:12.551647 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:12.551647 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:12.551647 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:12.551647 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:12.551647 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:12.551647 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:12.551647 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:12.551647 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:12.551647 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:12.551647 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:12.551647 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:12.551647 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:12.551647 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:12.553499 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:29:12.553499 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:29:12.553499 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:12.553499 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:12.553499 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 363/746] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:29:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:12.553499 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf17d18-1059-4440-9c2b-c07f2373134a None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:12.625793 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 384/747] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:29:12 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 16:29:12.626759 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8947ceea-ef56-4b62-a3ce-4a5cb34e1766 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:12.665556 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:12.692665 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf17d18-1059-4440-9c2b-c07f2373134a None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:12.735114 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab314bb-8d96-4db2-abdb-b4bb44a9e43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:12.736166 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab314bb-8d96-4db2-abdb-b4bb44a9e43a None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:12.736604 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab314bb-8d96-4db2-abdb-b4bb44a9e43a None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:12.737101 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab314bb-8d96-4db2-abdb-b4bb44a9e43a None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:12.738017 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab314bb-8d96-4db2-abdb-b4bb44a9e43a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:12.739622 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab314bb-8d96-4db2-abdb-b4bb44a9e43a None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:12.755141 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 364/748] 10.222.0.36 () {60 vars in 1141 bytes} [Thu May 19 16:29:12 2022] POST /identity/v3/services => generated 251 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:29:12.755799 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab314bb-8d96-4db2-abdb-b4bb44a9e43a None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:12.758112 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8947ceea-ef56-4b62-a3ce-4a5cb34e1766 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:12.813284 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a5c4a9-aa26-4ab6-be0c-e9c7859f3466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:12.814123 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a5c4a9-aa26-4ab6-be0c-e9c7859f3466 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:12.814412 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a5c4a9-aa26-4ab6-be0c-e9c7859f3466 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:12.814695 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a5c4a9-aa26-4ab6-be0c-e9c7859f3466 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:12.826613 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a5c4a9-aa26-4ab6-be0c-e9c7859f3466 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:12.828183 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a5c4a9-aa26-4ab6-be0c-e9c7859f3466 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:12.844939 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:12.853177 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-29a5c4a9-aa26-4ab6-be0c-e9c7859f3466 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:12.853177 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:12.853177 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:12.853177 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:12.853177 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:12.853177 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:12.853177 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:12.853177 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:12.853177 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:12.853177 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:12.853177 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:12.853177 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:12.853177 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:29:12.853177 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:29:12.853177 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:29:12.854159 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:29:12.854159 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:12.854159 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:12.854159 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:12.854159 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:12.854159 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:12.854159 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:12.854159 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:12.854159 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:12.854159 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:12.854159 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:12.854159 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:12.854159 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:12.854159 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:12.854159 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:12.854159 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:12.854159 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:12.855042 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:29:12.855042 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:29:12.855042 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:12.855042 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:12.855042 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 385/749] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:29:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:12.855334 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a5c4a9-aa26-4ab6-be0c-e9c7859f3466 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:12.933065 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab314bb-8d96-4db2-abdb-b4bb44a9e43a None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:12.968932 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:12.978664 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710e1cb0-6f0e-4cca-8cf5-8077ecc81866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:12.979759 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e1cb0-6f0e-4cca-8cf5-8077ecc81866 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:12.980207 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e1cb0-6f0e-4cca-8cf5-8077ecc81866 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:12.980634 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e1cb0-6f0e-4cca-8cf5-8077ecc81866 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:12.981987 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710e1cb0-6f0e-4cca-8cf5-8077ecc81866 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:12.981987 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710e1cb0-6f0e-4cca-8cf5-8077ecc81866 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:12.985404 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710e1cb0-6f0e-4cca-8cf5-8077ecc81866 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:13.001859 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 365/750] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:29:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:13.002154 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a5c4a9-aa26-4ab6-be0c-e9c7859f3466 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:13.002730 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710e1cb0-6f0e-4cca-8cf5-8077ecc81866 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:13.052486 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6854562-065d-4169-b5be-90d58e99e93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:13.053761 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6854562-065d-4169-b5be-90d58e99e93c None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:13.054259 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6854562-065d-4169-b5be-90d58e99e93c None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:13.054752 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6854562-065d-4169-b5be-90d58e99e93c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:13.071036 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6854562-065d-4169-b5be-90d58e99e93c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:13.072523 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6854562-065d-4169-b5be-90d58e99e93c None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:13.099109 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-d6854562-065d-4169-b5be-90d58e99e93c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:13.099109 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:13.099109 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:13.099109 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:13.099109 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:13.099109 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:13.099109 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:13.099109 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:13.099109 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:13.099109 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:13.099109 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:13.099109 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:13.099109 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:29:13.099109 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:29:13.099109 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:29:13.100083 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:29:13.100083 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:13.100083 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:13.100083 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:13.100083 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:13.100083 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:13.100083 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:13.100083 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:13.100083 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:13.100083 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:13.100083 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:13.100083 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:13.100083 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:13.100083 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:13.100083 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:13.100083 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:13.100083 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:13.100975 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:29:13.100975 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:29:13.100975 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:13.100975 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:13.102751 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 386/751] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:29:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:13.107430 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6854562-065d-4169-b5be-90d58e99e93c None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:13.139135 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710e1cb0-6f0e-4cca-8cf5-8077ecc81866 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:13.162201 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:13.191871 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9a9b74-ee8d-4ca6-98e9-e72f6b5b847d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:13.192902 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a9b74-ee8d-4ca6-98e9-e72f6b5b847d None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:13.193828 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a9b74-ee8d-4ca6-98e9-e72f6b5b847d None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:13.194383 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a9b74-ee8d-4ca6-98e9-e72f6b5b847d None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:13.195708 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9a9b74-ee8d-4ca6-98e9-e72f6b5b847d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:13.196484 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9a9b74-ee8d-4ca6-98e9-e72f6b5b847d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:13.198372 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9a9b74-ee8d-4ca6-98e9-e72f6b5b847d None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:13.219187 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 366/752] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:29:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:13.219187 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9a9b74-ee8d-4ca6-98e9-e72f6b5b847d None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:13.224000 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6854562-065d-4169-b5be-90d58e99e93c None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:13.254210 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:13.264197 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff1d4e3-eb21-46b5-bd37-a34b374ac52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:13.265315 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff1d4e3-eb21-46b5-bd37-a34b374ac52d None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:13.265840 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff1d4e3-eb21-46b5-bd37-a34b374ac52d None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:13.266300 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff1d4e3-eb21-46b5-bd37-a34b374ac52d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:13.274245 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff1d4e3-eb21-46b5-bd37-a34b374ac52d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:13.275985 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff1d4e3-eb21-46b5-bd37-a34b374ac52d None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:13.290720 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-4ff1d4e3-eb21-46b5-bd37-a34b374ac52d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 16:29:13.290720 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:13.290720 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:13.290720 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:13.290720 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:13.290720 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:13.290720 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:13.290720 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:13.290720 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:13.290720 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:13.290720 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:13.290720 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:13.290720 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 16:29:13.290720 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 16:29:13.290720 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 16:29:13.290720 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 16:29:13.291689 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:13.291689 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:13.291689 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 16:29:13.291689 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:29:13.291689 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 16:29:13.291689 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:29:13.291689 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:13.291689 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:13.291689 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:13.291689 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:13.291689 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:13.291689 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:13.291689 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:13.291689 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:13.291689 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:13.291689 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:13.291689 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:13.291689 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:13.292624 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:13.292624 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:13.292624 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 16:29:13.292624 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 16:29:13.292624 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 16:29:13.292624 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 16:29:13.292624 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 16:29:13.292624 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 16:29:13.292624 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 16:29:13.292624 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:13.295197 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 387/753] 10.222.0.36 () {56 vars in 1093 bytes} [Thu May 19 16:29:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:13.296199 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff1d4e3-eb21-46b5-bd37-a34b374ac52d None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:13.336483 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9a9b74-ee8d-4ca6-98e9-e72f6b5b847d None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:13.375666 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075609f2-4131-4eb6-a996-9573e21bf742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:13.376581 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075609f2-4131-4eb6-a996-9573e21bf742 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:13.376767 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075609f2-4131-4eb6-a996-9573e21bf742 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:13.377021 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075609f2-4131-4eb6-a996-9573e21bf742 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:13.377800 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075609f2-4131-4eb6-a996-9573e21bf742 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:13.378112 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075609f2-4131-4eb6-a996-9573e21bf742 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:13.379693 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075609f2-4131-4eb6-a996-9573e21bf742 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:13.397586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 367/754] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:29:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:13.397586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075609f2-4131-4eb6-a996-9573e21bf742 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:13.408984 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff1d4e3-eb21-46b5-bd37-a34b374ac52d None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:13.436647 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:13.445477 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6055d3-fd3b-4827-8fc7-75b44a50ca6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:13.446584 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6055d3-fd3b-4827-8fc7-75b44a50ca6e None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:13.447054 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6055d3-fd3b-4827-8fc7-75b44a50ca6e None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:13.447511 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6055d3-fd3b-4827-8fc7-75b44a50ca6e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:13.454949 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6055d3-fd3b-4827-8fc7-75b44a50ca6e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:13.456732 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6055d3-fd3b-4827-8fc7-75b44a50ca6e None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:13.471950 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-2d6055d3-fd3b-4827-8fc7-75b44a50ca6e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:29:13.471950 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:13.471950 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:13.471950 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:13.471950 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:13.471950 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:13.471950 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:13.471950 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:13.471950 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:13.471950 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:13.471950 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:13.471950 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:13.471950 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 16:29:13.471950 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 16:29:13.471950 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 16:29:13.472875 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 16:29:13.472875 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:13.472875 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:13.472875 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:13.472875 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:13.472875 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:13.472875 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:13.472875 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:13.472875 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:13.472875 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:13.472875 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:13.472875 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:13.472875 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:13.472875 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:13.472875 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:13.472875 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:13.472875 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:13.473786 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 16:29:13.473786 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 16:29:13.473786 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 16:29:13.473786 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 16:29:13.473786 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 16:29:13.473786 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 16:29:13.473786 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:29:13.473786 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:13.480679 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 388/755] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:29:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:13.481841 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6055d3-fd3b-4827-8fc7-75b44a50ca6e None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:13.513246 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075609f2-4131-4eb6-a996-9573e21bf742 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:13.531139 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:13.555861 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bf543b-9ad2-4256-805b-ec77d931e7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:13.556657 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bf543b-9ad2-4256-805b-ec77d931e7f9 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:13.557026 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bf543b-9ad2-4256-805b-ec77d931e7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:13.557315 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bf543b-9ad2-4256-805b-ec77d931e7f9 None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:13.558194 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bf543b-9ad2-4256-805b-ec77d931e7f9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:13.558508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bf543b-9ad2-4256-805b-ec77d931e7f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:13.560120 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bf543b-9ad2-4256-805b-ec77d931e7f9 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:13.587493 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 368/756] 10.222.0.36 () {56 vars in 1133 bytes} [Thu May 19 16:29:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:13.588604 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bf543b-9ad2-4256-805b-ec77d931e7f9 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:13.595621 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6055d3-fd3b-4827-8fc7-75b44a50ca6e None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:13.633296 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c93438-91a7-4bcb-b286-bb82b8a8f922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:13.634512 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c93438-91a7-4bcb-b286-bb82b8a8f922 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:13.634999 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c93438-91a7-4bcb-b286-bb82b8a8f922 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:13.635474 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c93438-91a7-4bcb-b286-bb82b8a8f922 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:13.643068 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c93438-91a7-4bcb-b286-bb82b8a8f922 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:13.645093 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c93438-91a7-4bcb-b286-bb82b8a8f922 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:13.661013 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-22c93438-91a7-4bcb-b286-bb82b8a8f922 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:29:13.661013 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:13.661013 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:13.661013 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:13.661013 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:13.661013 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:13.661013 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:13.661013 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:13.661013 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:13.661013 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:13.661013 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:13.661013 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:13.661013 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 16:29:13.661013 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 16:29:13.661013 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 16:29:13.662115 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 16:29:13.662115 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:13.662115 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:13.662115 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:13.662115 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:13.662115 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:13.662115 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:13.662115 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:13.662115 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:13.662115 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:13.662115 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:13.662115 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:13.662115 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:13.662115 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:13.662115 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:13.662115 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:13.662115 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:13.663005 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 16:29:13.663005 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 16:29:13.663005 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 16:29:13.663005 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 16:29:13.663005 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 16:29:13.663005 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 16:29:13.663005 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:29:13.663005 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:13.669532 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 389/757] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:29:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:13.670703 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c93438-91a7-4bcb-b286-bb82b8a8f922 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:13.694471 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:13.708815 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bf543b-9ad2-4256-805b-ec77d931e7f9 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:13.750935 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d022aba1-cb9d-402c-a7f9-2fd5e0770f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:13.751614 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d022aba1-cb9d-402c-a7f9-2fd5e0770f29 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:13.751904 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d022aba1-cb9d-402c-a7f9-2fd5e0770f29 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:13.752183 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d022aba1-cb9d-402c-a7f9-2fd5e0770f29 None admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:13.752992 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d022aba1-cb9d-402c-a7f9-2fd5e0770f29 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:13.753336 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d022aba1-cb9d-402c-a7f9-2fd5e0770f29 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:13.755015 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d022aba1-cb9d-402c-a7f9-2fd5e0770f29 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:13.773997 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 369/758] 10.222.0.36 () {56 vars in 1148 bytes} [Thu May 19 16:29:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:13.773997 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d022aba1-cb9d-402c-a7f9-2fd5e0770f29 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:13.786980 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c93438-91a7-4bcb-b286-bb82b8a8f922 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:13.795519 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:13.825398 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d14b3-6224-4915-820e-0a69da0e0741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:13.826622 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d14b3-6224-4915-820e-0a69da0e0741 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:13.827156 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d14b3-6224-4915-820e-0a69da0e0741 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:13.827631 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d14b3-6224-4915-820e-0a69da0e0741 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:13.839390 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9d14b3-6224-4915-820e-0a69da0e0741 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:13.841244 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9d14b3-6224-4915-820e-0a69da0e0741 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:13.860552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-ee9d14b3-6224-4915-820e-0a69da0e0741 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:13.860552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:13.860552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:13.860552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:13.860552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:13.860552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:13.860552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:13.860552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:13.860552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:13.860552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:13.860552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:13.860552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:13.860552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:29:13.860552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:29:13.860552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:29:13.861532 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:29:13.861532 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:13.861532 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:13.861532 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:13.861532 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:13.861532 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:13.861532 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:13.861532 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:13.861532 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:13.861532 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:13.861532 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:13.861532 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:13.861532 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:13.861532 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:13.861532 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:13.861532 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:13.861532 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:13.862423 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:29:13.862423 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:29:13.862423 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:13.862423 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:13.864803 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 390/759] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:29:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:13.866006 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d14b3-6224-4915-820e-0a69da0e0741 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:13.898324 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d022aba1-cb9d-402c-a7f9-2fd5e0770f29 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:13.940121 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21b07c3-b297-49f1-80b4-6e776b88ba8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:13.940886 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b07c3-b297-49f1-80b4-6e776b88ba8b None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:13.941131 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b07c3-b297-49f1-80b4-6e776b88ba8b None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:13.941442 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b07c3-b297-49f1-80b4-6e776b88ba8b None admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:13.942213 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21b07c3-b297-49f1-80b4-6e776b88ba8b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:13.942527 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21b07c3-b297-49f1-80b4-6e776b88ba8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:13.944165 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21b07c3-b297-49f1-80b4-6e776b88ba8b None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:13.963217 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 370/760] 10.222.0.36 () {56 vars in 1148 bytes} [Thu May 19 16:29:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:13.963217 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21b07c3-b297-49f1-80b4-6e776b88ba8b None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:13.976014 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:13.994770 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d14b3-6224-4915-820e-0a69da0e0741 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:14.034431 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f548e1a3-94c3-416e-bf9c-fd14e76907e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:14.035610 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548e1a3-94c3-416e-bf9c-fd14e76907e1 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:14.036131 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548e1a3-94c3-416e-bf9c-fd14e76907e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:14.036608 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548e1a3-94c3-416e-bf9c-fd14e76907e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:14.037691 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f548e1a3-94c3-416e-bf9c-fd14e76907e1 None admin] RBAC: Adding query filter params (role.id=e2915fe109244a07a9be96bd2b219fdd, scope.project.id=631c76794a124d6da22c626e590e4352, user.id=ffe7cebc6f3b40d8a8e1866ed2da9822) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:14.038223 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f548e1a3-94c3-416e-bf9c-fd14e76907e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:14.039894 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f548e1a3-94c3-416e-bf9c-fd14e76907e1 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:14.051752 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 391/761] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:29:13 2022] GET /identity/v3/role_assignments?user.id=ffe7cebc6f3b40d8a8e1866ed2da9822&scope.project.id=631c76794a124d6da22c626e590e4352&role.id=e2915fe109244a07a9be96bd2b219fdd => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:14.053052 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f548e1a3-94c3-416e-bf9c-fd14e76907e1 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:14.077943 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21b07c3-b297-49f1-80b4-6e776b88ba8b None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:14.090948 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:14.123612 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312dd58a-332d-4be3-bc64-46c5fe73fb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:14.124363 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312dd58a-332d-4be3-bc64-46c5fe73fb29 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:14.124735 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312dd58a-332d-4be3-bc64-46c5fe73fb29 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:14.125584 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312dd58a-332d-4be3-bc64-46c5fe73fb29 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:14.126343 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312dd58a-332d-4be3-bc64-46c5fe73fb29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:14.126658 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312dd58a-332d-4be3-bc64-46c5fe73fb29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:14.128095 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312dd58a-332d-4be3-bc64-46c5fe73fb29 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:14.144437 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 371/762] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:29:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:14.150593 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b4e4b-e06b-4755-a142-58f0375bd820 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:14.150897 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b4e4b-e06b-4755-a142-58f0375bd820 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:14.151181 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b4e4b-e06b-4755-a142-58f0375bd820 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:14.152948 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 372/763] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:14 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:29:14.153788 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011b4e4b-e06b-4755-a142-58f0375bd820 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:14.197706 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f548e1a3-94c3-416e-bf9c-fd14e76907e1 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:14.241483 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac85ae41-51e2-47ea-8ac9-552e8d43fac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:14.242588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85ae41-51e2-47ea-8ac9-552e8d43fac8 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:14.243104 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85ae41-51e2-47ea-8ac9-552e8d43fac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:14.243656 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85ae41-51e2-47ea-8ac9-552e8d43fac8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:14.244721 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac85ae41-51e2-47ea-8ac9-552e8d43fac8 None admin] RBAC: Adding query filter params (role.id=727026b648e1429ca3697e04800b328a, scope.project.id=631c76794a124d6da22c626e590e4352, user.id=a30dc1dd2b00463bb3c7957864c56043) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:14.245309 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac85ae41-51e2-47ea-8ac9-552e8d43fac8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:14.247075 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac85ae41-51e2-47ea-8ac9-552e8d43fac8 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:14.265269 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 392/764] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:29:14 2022] GET /identity/v3/role_assignments?user.id=a30dc1dd2b00463bb3c7957864c56043&scope.project.id=631c76794a124d6da22c626e590e4352&role.id=727026b648e1429ca3697e04800b328a => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:14.267448 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343eb00b-e794-44a7-a5ff-a50c7b005c5c None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:14.267744 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343eb00b-e794-44a7-a5ff-a50c7b005c5c None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:14.268032 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343eb00b-e794-44a7-a5ff-a50c7b005c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:14.271639 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:14.288497 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011b4e4b-e06b-4755-a142-58f0375bd820 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:14.291622 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-343eb00b-e794-44a7-a5ff-a50c7b005c5c None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:14.292867 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-343eb00b-e794-44a7-a5ff-a50c7b005c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:14.337720 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f43924-643c-43b7-ad7d-4951cda8fa88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:14.338537 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f43924-643c-43b7-ad7d-4951cda8fa88 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:14.338823 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f43924-643c-43b7-ad7d-4951cda8fa88 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:14.339099 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f43924-643c-43b7-ad7d-4951cda8fa88 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:14.346761 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f43924-643c-43b7-ad7d-4951cda8fa88 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:14.348282 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f43924-643c-43b7-ad7d-4951cda8fa88 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:14.364120 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-69f43924-643c-43b7-ad7d-4951cda8fa88 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:29:14.364120 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:14.364120 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:14.364120 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:14.364120 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:14.364120 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:14.364120 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:14.364120 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:14.364120 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:14.364120 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:14.364120 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:14.364120 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:14.364120 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 16:29:14.364120 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 16:29:14.364120 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 16:29:14.365081 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 16:29:14.365081 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:14.365081 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:14.365081 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:14.365081 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:14.365081 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:14.365081 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:14.365081 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:14.365081 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:14.365081 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:14.365081 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:14.365081 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:14.365081 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:14.365081 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:14.365081 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:14.365081 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:14.365081 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:14.366024 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 16:29:14.366024 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 16:29:14.366024 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 16:29:14.366024 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 16:29:14.366024 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 16:29:14.366024 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 16:29:14.366024 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:29:14.366024 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:14.366024 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 373/765] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:29:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:14.373209 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f43924-643c-43b7-ad7d-4951cda8fa88 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:14.463020 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 393/766] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:29:14.507574 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343eb00b-e794-44a7-a5ff-a50c7b005c5c None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:14.518382 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f43924-643c-43b7-ad7d-4951cda8fa88 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:14.528337 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:14.553490 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:14.558927 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5854cc2a-a101-47ec-a620-6a4b55a5fe51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:14.560061 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5854cc2a-a101-47ec-a620-6a4b55a5fe51 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:14.560558 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5854cc2a-a101-47ec-a620-6a4b55a5fe51 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:14.561037 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5854cc2a-a101-47ec-a620-6a4b55a5fe51 None admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:14.562070 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5854cc2a-a101-47ec-a620-6a4b55a5fe51 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:14.563166 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5854cc2a-a101-47ec-a620-6a4b55a5fe51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:14.564844 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5854cc2a-a101-47ec-a620-6a4b55a5fe51 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:14.577950 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 374/767] 10.222.0.36 () {56 vars in 1148 bytes} [Thu May 19 16:29:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:14.584668 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5854cc2a-a101-47ec-a620-6a4b55a5fe51 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:14.652354 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343eb00b-e794-44a7-a5ff-a50c7b005c5c None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:14.724809 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86e80e-5c2c-45c7-81b7-e0ffb7ae0c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:14.726058 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86e80e-5c2c-45c7-81b7-e0ffb7ae0c43 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:14.726543 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86e80e-5c2c-45c7-81b7-e0ffb7ae0c43 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:14.727016 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86e80e-5c2c-45c7-81b7-e0ffb7ae0c43 None admin] PATH_INFO: `/v3/services/network` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:14.727883 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b86e80e-5c2c-45c7-81b7-e0ffb7ae0c43 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:14.732903 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b86e80e-5c2c-45c7-81b7-e0ffb7ae0c43 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:14.761262 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-3b86e80e-5c2c-45c7-81b7-e0ffb7ae0c43 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 16:29:14.761262 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:14.761262 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 16:29:14.761262 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 16:29:14.761262 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 16:29:14.761262 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 16:29:14.761262 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 16:29:14.761262 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:29:14.761262 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 16:29:14.761262 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:14.761262 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 16:29:14.761262 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:14.761262 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:14.761262 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:14.761262 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:14.761262 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:14.761262 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:14.762259 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:14.762259 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:14.762259 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:14.762259 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:14.762259 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:14.762259 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:14.762259 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 16:29:14.762259 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 16:29:14.762259 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 16:29:14.762259 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 16:29:14.762259 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:14.762259 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:14.762259 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:14.762259 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:14.762259 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:14.762259 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:14.763103 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:14.763103 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:14.763103 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:14.763103 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:14.763103 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:14.763103 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:14.763103 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:14.763103 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:14.763103 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:14.763103 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:14.763103 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 16:29:14.763103 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:29:14.763103 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 16:29:14.763103 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:14.763835 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5854cc2a-a101-47ec-a620-6a4b55a5fe51 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:14.766293 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 394/768] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:29:14 2022] GET /identity/v3/services/network => generated 88 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:14.773136 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86e80e-5c2c-45c7-81b7-e0ffb7ae0c43 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:14.825894 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c218ce-140b-4004-a30a-b813978c437e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:14.826814 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c218ce-140b-4004-a30a-b813978c437e None admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:14.827098 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c218ce-140b-4004-a30a-b813978c437e None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:14.827378 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c218ce-140b-4004-a30a-b813978c437e None admin] PATH_INFO: `/v3/projects/631c76794a124d6da22c626e590e4352/users/58bdeada00b74b7b8e186e6dc51e6dfb/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:14.833995 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c218ce-140b-4004-a30a-b813978c437e None admin] RBAC: Authorizing `identity:create_grant(project_id=631c76794a124d6da22c626e590e4352, user_id=58bdeada00b74b7b8e186e6dc51e6dfb, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:14.835458 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c218ce-140b-4004-a30a-b813978c437e None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:14.859135 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 375/769] 10.222.0.36 () {58 vars in 1439 bytes} [Thu May 19 16:29:14 2022] PUT /identity/v3/projects/631c76794a124d6da22c626e590e4352/users/58bdeada00b74b7b8e186e6dc51e6dfb/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:29:14.898319 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:14.914625 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86e80e-5c2c-45c7-81b7-e0ffb7ae0c43 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:14.973518 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a907b744-a559-42d3-a4d6-9aa38c275349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:14.974236 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a907b744-a559-42d3-a4d6-9aa38c275349 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:14.974520 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a907b744-a559-42d3-a4d6-9aa38c275349 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:14.974798 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a907b744-a559-42d3-a4d6-9aa38c275349 None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:14.975568 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a907b744-a559-42d3-a4d6-9aa38c275349 None admin] RBAC: Adding query filter params (name=network) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:14.975902 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a907b744-a559-42d3-a4d6-9aa38c275349 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:14.977524 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a907b744-a559-42d3-a4d6-9aa38c275349 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:14.983669 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 395/770] 10.222.0.36 () {56 vars in 1112 bytes} [Thu May 19 16:29:14 2022] GET /identity/v3/services?name=network => generated 124 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:29:14.990186 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c218ce-140b-4004-a30a-b813978c437e None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:15.056959 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:15.116709 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c218ce-140b-4004-a30a-b813978c437e None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:15.158235 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa9da06-3c2c-4335-a197-808153b2a2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:15.158917 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9da06-3c2c-4335-a197-808153b2a2c1 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:15.159204 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9da06-3c2c-4335-a197-808153b2a2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:15.159482 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9da06-3c2c-4335-a197-808153b2a2c1 None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:15.160234 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa9da06-3c2c-4335-a197-808153b2a2c1 None admin] RBAC: Adding query filter params (type=network) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:15.160541 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa9da06-3c2c-4335-a197-808153b2a2c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:15.161986 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa9da06-3c2c-4335-a197-808153b2a2c1 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:15.169079 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 376/771] 10.222.0.36 () {56 vars in 1112 bytes} [Thu May 19 16:29:14 2022] GET /identity/v3/services?type=network => generated 350 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:15.175425 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a907b744-a559-42d3-a4d6-9aa38c275349 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:15.208624 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387be08-93d2-44b7-8969-0f75ba5986dd None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:15.208949 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387be08-93d2-44b7-8969-0f75ba5986dd None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:15.209266 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387be08-93d2-44b7-8969-0f75ba5986dd None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:15.210948 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 377/772] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 16:29:15.216923 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4c942-b7fc-46b3-ad1e-9d95fd33b9b3 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:15.217257 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4c942-b7fc-46b3-ad1e-9d95fd33b9b3 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:15.217582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4c942-b7fc-46b3-ad1e-9d95fd33b9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:15.243559 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-a8f4c942-b7fc-46b3-ad1e-9d95fd33b9b3 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:15.244779 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-a8f4c942-b7fc-46b3-ad1e-9d95fd33b9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:15.356298 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a907b744-a559-42d3-a4d6-9aa38c275349 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:15.393703 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c067e92-8bb9-4979-9097-9a32bdeabe3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:15.394398 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c067e92-8bb9-4979-9097-9a32bdeabe3b None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:15.394944 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c067e92-8bb9-4979-9097-9a32bdeabe3b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:15.395231 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c067e92-8bb9-4979-9097-9a32bdeabe3b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:15.396051 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c067e92-8bb9-4979-9097-9a32bdeabe3b None admin] RBAC: Adding query filter params (interface=public, service_id=5814c3a8a8fb4b76831a445ce42e120c, region_id=RegionOne) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:15.396373 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c067e92-8bb9-4979-9097-9a32bdeabe3b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:15.397813 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c067e92-8bb9-4979-9097-9a32bdeabe3b None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:15.407481 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 396/773] 10.222.0.36 () {56 vars in 1251 bytes} [Thu May 19 16:29:15 2022] GET /identity/v3/endpoints?service_id=5814c3a8a8fb4b76831a445ce42e120c&interface=public®ion_id=RegionOne => generated 194 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:15.412446 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c067e92-8bb9-4979-9097-9a32bdeabe3b None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:15.426680 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:15.474935 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 378/774] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 16:29:15.514257 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f4c942-b7fc-46b3-ad1e-9d95fd33b9b3 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:15.525927 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c067e92-8bb9-4979-9097-9a32bdeabe3b None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:15.564151 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccbd0bd-16cb-40e3-96b4-97507b344abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:15.565398 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccbd0bd-16cb-40e3-96b4-97507b344abb None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:15.565790 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccbd0bd-16cb-40e3-96b4-97507b344abb None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:15.566182 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccbd0bd-16cb-40e3-96b4-97507b344abb None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:15.567041 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccbd0bd-16cb-40e3-96b4-97507b344abb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:15.568541 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccbd0bd-16cb-40e3-96b4-97507b344abb None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:15.574911 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 397/775] 10.222.0.36 () {56 vars in 1087 bytes} [Thu May 19 16:29:15 2022] GET /identity/v3/services => generated 772 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:29:15.621537 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:15.699801 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f4c942-b7fc-46b3-ad1e-9d95fd33b9b3 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:15.757176 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80076df-f728-45a0-9894-2679cfc06cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:15.758034 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80076df-f728-45a0-9894-2679cfc06cf4 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:15.758321 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80076df-f728-45a0-9894-2679cfc06cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:15.758603 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80076df-f728-45a0-9894-2679cfc06cf4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:15.759300 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80076df-f728-45a0-9894-2679cfc06cf4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:15.760741 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80076df-f728-45a0-9894-2679cfc06cf4 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:15.781805 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-e80076df-f728-45a0-9894-2679cfc06cf4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 16:29:15.781805 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:15.781805 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 16:29:15.781805 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 16:29:15.781805 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 16:29:15.781805 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 16:29:15.781805 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 16:29:15.781805 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:29:15.781805 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 16:29:15.781805 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:15.781805 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 16:29:15.781805 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:15.781805 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:15.781805 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:15.781805 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:15.781805 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:15.783371 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:15.783371 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:15.783371 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:15.783371 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:15.783371 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:15.783371 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:15.783371 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:15.783371 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 16:29:15.783371 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 16:29:15.783371 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 16:29:15.783371 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 16:29:15.783371 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:15.783371 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:15.783371 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:15.783371 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:15.783371 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:15.784763 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:15.784763 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:15.784763 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:15.784763 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:15.784763 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:15.784763 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:15.784763 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:15.784763 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:15.784763 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:15.784763 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:15.784763 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:15.784763 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 16:29:15.784763 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:29:15.784763 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 16:29:15.784763 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:15.784763 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 379/776] 10.222.0.36 () {56 vars in 1129 bytes} [Thu May 19 16:29:15 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:15.793610 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccbd0bd-16cb-40e3-96b4-97507b344abb None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:15.915845 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:15.936773 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccbd0bd-16cb-40e3-96b4-97507b344abb None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:15.999037 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e6ecc3-7b22-43ef-9ef7-956a41f3405e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:15.999756 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6ecc3-7b22-43ef-9ef7-956a41f3405e None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:16.000608 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6ecc3-7b22-43ef-9ef7-956a41f3405e None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:16.000895 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6ecc3-7b22-43ef-9ef7-956a41f3405e None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:16.001665 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e6ecc3-7b22-43ef-9ef7-956a41f3405e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:16.001980 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e6ecc3-7b22-43ef-9ef7-956a41f3405e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:16.003412 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e6ecc3-7b22-43ef-9ef7-956a41f3405e None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:16.013244 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 398/777] 10.222.0.36 () {56 vars in 1124 bytes} [Thu May 19 16:29:15 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:16.022261 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80076df-f728-45a0-9894-2679cfc06cf4 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:16.134116 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80076df-f728-45a0-9894-2679cfc06cf4 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:16.148569 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:16.187776 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba203018-caa9-4bb4-88af-10a7f4a2a2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:16.188461 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba203018-caa9-4bb4-88af-10a7f4a2a2a4 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:16.188753 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba203018-caa9-4bb4-88af-10a7f4a2a2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:16.189039 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba203018-caa9-4bb4-88af-10a7f4a2a2a4 None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:16.189809 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba203018-caa9-4bb4-88af-10a7f4a2a2a4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:16.190131 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba203018-caa9-4bb4-88af-10a7f4a2a2a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:16.191524 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba203018-caa9-4bb4-88af-10a7f4a2a2a4 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:16.201693 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 380/778] 10.222.0.36 () {56 vars in 1124 bytes} [Thu May 19 16:29:16 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:16.206272 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e6ecc3-7b22-43ef-9ef7-956a41f3405e None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:16.400203 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e6ecc3-7b22-43ef-9ef7-956a41f3405e None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:16.453698 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b4b189-da72-4dfb-83b4-1e72b8ca15dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:16.454390 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4b189-da72-4dfb-83b4-1e72b8ca15dc None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:16.454680 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4b189-da72-4dfb-83b4-1e72b8ca15dc None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:16.454964 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4b189-da72-4dfb-83b4-1e72b8ca15dc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:16.455789 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b4b189-da72-4dfb-83b4-1e72b8ca15dc None admin] RBAC: Adding query filter params (interface=public, service_id=9c570481c91b42529dbd046863cdfbeb, region_id=RegionOne) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:16.456105 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b4b189-da72-4dfb-83b4-1e72b8ca15dc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:16.457538 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b4b189-da72-4dfb-83b4-1e72b8ca15dc None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:16.469792 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:16.470088 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 399/779] 10.222.0.36 () {56 vars in 1251 bytes} [Thu May 19 16:29:16 2022] GET /identity/v3/endpoints?service_id=9c570481c91b42529dbd046863cdfbeb&interface=public®ion_id=RegionOne => generated 194 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:16.475235 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba203018-caa9-4bb4-88af-10a7f4a2a2a4 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:16.609214 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba203018-caa9-4bb4-88af-10a7f4a2a2a4 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:16.645308 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3534eb-94af-4037-8ac1-496da4723c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:16.645989 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3534eb-94af-4037-8ac1-496da4723c7a None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:16.646277 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3534eb-94af-4037-8ac1-496da4723c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:16.646554 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3534eb-94af-4037-8ac1-496da4723c7a None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:16.647247 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3534eb-94af-4037-8ac1-496da4723c7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:16.648628 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3534eb-94af-4037-8ac1-496da4723c7a None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:16.654550 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 381/780] 10.222.0.36 () {56 vars in 1087 bytes} [Thu May 19 16:29:16 2022] GET /identity/v3/services => generated 772 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:16.946970 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6d65f-9488-43ce-ae95-323e306cd854 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:16.947286 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6d65f-9488-43ce-ae95-323e306cd854 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:16.947476 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6d65f-9488-43ce-ae95-323e306cd854 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:16.948955 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 400/781] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:29:16.957834 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b8b5bf-cd40-43d4-a8c7-5dbcd502b245 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:16.958158 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b8b5bf-cd40-43d4-a8c7-5dbcd502b245 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:16.958410 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b8b5bf-cd40-43d4-a8c7-5dbcd502b245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:16.981125 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-f7b8b5bf-cd40-43d4-a8c7-5dbcd502b245 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:16.982922 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-f7b8b5bf-cd40-43d4-a8c7-5dbcd502b245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:17.023695 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:17.070584 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 382/782] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:29:17.096201 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a6d65f-9488-43ce-ae95-323e306cd854 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:17.173958 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2b911-b42a-4dee-8d25-a7873bc4af05 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:17.174625 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2b911-b42a-4dee-8d25-a7873bc4af05 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:17.175117 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2b911-b42a-4dee-8d25-a7873bc4af05 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:17.178132 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 383/783] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 16:29:17.182759 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e94df-9a9e-4f98-92d9-ccad5f517e7b None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:17.183321 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e94df-9a9e-4f98-92d9-ccad5f517e7b None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:17.183810 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e94df-9a9e-4f98-92d9-ccad5f517e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:17.201914 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-632e94df-9a9e-4f98-92d9-ccad5f517e7b None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:17.203501 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-632e94df-9a9e-4f98-92d9-ccad5f517e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:17.290224 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a6d65f-9488-43ce-ae95-323e306cd854 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:17.320538 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 384/784] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:29:17.346452 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c7e0c-5c32-43f4-bb0e-f63c0197d2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:17.347289 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c7e0c-5c32-43f4-bb0e-f63c0197d2a4 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:17.347637 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c7e0c-5c32-43f4-bb0e-f63c0197d2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:17.347921 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c7e0c-5c32-43f4-bb0e-f63c0197d2a4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:17.366611 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9c7e0c-5c32-43f4-bb0e-f63c0197d2a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:29:17.373629 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632e94df-9a9e-4f98-92d9-ccad5f517e7b None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:17.384190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-fc9c7e0c-5c32-43f4-bb0e-f63c0197d2a4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:29:17.384190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:17.384190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:17.384190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:17.384190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:17.384190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:17.384190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:17.384190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:17.384190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:17.384190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:17.384190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:17.384190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:17.384190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:29:17.384190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:29:17.384190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:29:17.384190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:29:17.385260 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:29:17.385260 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:29:17.385260 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:29:17.385260 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:29:17.385260 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:17.385260 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:17.385260 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:17.385260 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:17.385260 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:17.385260 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:17.385260 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:17.385260 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:17.385260 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:17.385260 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:17.385260 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:17.385260 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:17.386108 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:17.386108 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:17.386108 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:17.386108 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:17.386108 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:29:17.386108 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:29:17.386108 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:29:17.386108 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:29:17.386108 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:29:17.386108 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:29:17.386108 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:29:17.386108 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:17.393406 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 401/785] 10.222.0.36 () {56 vars in 1096 bytes} [Thu May 19 16:29:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:17.396369 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c7e0c-5c32-43f4-bb0e-f63c0197d2a4 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:17.511604 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:17.557684 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c7e0c-5c32-43f4-bb0e-f63c0197d2a4 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:17.568446 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632e94df-9a9e-4f98-92d9-ccad5f517e7b None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:17.570359 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:17.605317 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0890f7-7b22-48c2-bdcf-cc63cde7c610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:17.606254 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0890f7-7b22-48c2-bdcf-cc63cde7c610 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:17.606543 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0890f7-7b22-48c2-bdcf-cc63cde7c610 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:17.607239 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0890f7-7b22-48c2-bdcf-cc63cde7c610 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:17.608398 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0890f7-7b22-48c2-bdcf-cc63cde7c610 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:17.608716 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0890f7-7b22-48c2-bdcf-cc63cde7c610 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:17.610140 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0890f7-7b22-48c2-bdcf-cc63cde7c610 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:17.611851 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc88505-743b-4b63-b9b0-c67b3e7247d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:17.612656 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc88505-743b-4b63-b9b0-c67b3e7247d6 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:17.612958 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc88505-743b-4b63-b9b0-c67b3e7247d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:17.613354 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc88505-743b-4b63-b9b0-c67b3e7247d6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:17.621301 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 402/786] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:29:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:17.626657 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c931ba-1c61-4425-9c3b-713a62753a68 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:17.626947 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c931ba-1c61-4425-9c3b-713a62753a68 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:17.627225 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c931ba-1c61-4425-9c3b-713a62753a68 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:17.628695 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 403/787] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:29:17.629271 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c931ba-1c61-4425-9c3b-713a62753a68 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:17.630496 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc88505-743b-4b63-b9b0-c67b3e7247d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:29:17.638696 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-bcc88505-743b-4b63-b9b0-c67b3e7247d6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 16:29:17.638696 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:17.638696 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:17.638696 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:17.638696 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:17.638696 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:17.638696 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:17.638696 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:17.638696 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:17.638696 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:17.638696 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:17.638696 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:17.638696 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:29:17.638696 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:29:17.638696 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:29:17.638696 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:29:17.639678 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:29:17.639678 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:29:17.639678 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:29:17.639678 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:29:17.639678 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:17.639678 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:17.639678 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:17.639678 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:17.639678 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:17.639678 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:17.639678 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:17.639678 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:17.639678 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:17.639678 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:17.639678 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:17.639678 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:17.641090 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:17.641090 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:17.641090 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:17.641090 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:17.641090 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:29:17.641090 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:29:17.641090 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:29:17.641090 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:29:17.641090 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:29:17.641090 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:29:17.641090 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 16:29:17.641090 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:17.643859 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 385/788] 10.222.0.36 () {56 vars in 1102 bytes} [Thu May 19 16:29:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 16:29:17.645632 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dda11d-329d-4589-b633-a80b4486d0c6 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:17.646136 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dda11d-329d-4589-b633-a80b4486d0c6 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:17.646617 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dda11d-329d-4589-b633-a80b4486d0c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:17.664622 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-64dda11d-329d-4589-b633-a80b4486d0c6 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:17.666198 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-64dda11d-329d-4589-b633-a80b4486d0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:17.743075 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 386/789] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:29:17.744063 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dda11d-329d-4589-b633-a80b4486d0c6 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:17.760175 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c931ba-1c61-4425-9c3b-713a62753a68 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:17.786848 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:17.800126 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf1667d-2767-415a-ba7b-9deb6c0d9019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:17.801461 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1667d-2767-415a-ba7b-9deb6c0d9019 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:17.801968 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1667d-2767-415a-ba7b-9deb6c0d9019 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:17.802446 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1667d-2767-415a-ba7b-9deb6c0d9019 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:17.814790 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf1667d-2767-415a-ba7b-9deb6c0d9019 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:17.816593 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf1667d-2767-415a-ba7b-9deb6c0d9019 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:17.841399 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-dcf1667d-2767-415a-ba7b-9deb6c0d9019 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:17.841399 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:17.841399 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:17.841399 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:17.841399 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:17.841399 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:17.841399 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:17.841399 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:17.841399 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:17.841399 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:17.841399 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:17.841399 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:17.841399 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:29:17.841399 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:29:17.841399 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:29:17.842344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:29:17.842344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:17.842344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:17.842344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:17.842344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:17.842344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:17.842344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:17.842344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:17.842344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:17.842344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:17.842344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:17.842344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:17.842344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:17.842344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:17.842344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:17.842344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:17.842344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:17.843242 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:29:17.843242 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:29:17.843242 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:17.843242 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:17.846301 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 404/790] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:29:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:17.851542 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:17.852866 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf1667d-2767-415a-ba7b-9deb6c0d9019 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:17.862737 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dda11d-329d-4589-b633-a80b4486d0c6 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:17.902173 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae185e1-f50c-4f60-a512-16f15e4b3450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:17.903327 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae185e1-f50c-4f60-a512-16f15e4b3450 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:17.903480 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae185e1-f50c-4f60-a512-16f15e4b3450 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:17.903854 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae185e1-f50c-4f60-a512-16f15e4b3450 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:17.904732 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae185e1-f50c-4f60-a512-16f15e4b3450 None admin] RBAC: Adding query filter params (name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:17.905164 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae185e1-f50c-4f60-a512-16f15e4b3450 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:17.907221 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae185e1-f50c-4f60-a512-16f15e4b3450 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:17.921334 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 387/791] 10.222.0.36 () {56 vars in 1103 bytes} [Thu May 19 16:29:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:17.925492 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae185e1-f50c-4f60-a512-16f15e4b3450 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:17.987876 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf1667d-2767-415a-ba7b-9deb6c0d9019 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:18.026829 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c953ffc-cbd0-4b4e-8f4c-6919ef0b80f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:18.028192 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c953ffc-cbd0-4b4e-8f4c-6919ef0b80f0 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:18.028779 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c953ffc-cbd0-4b4e-8f4c-6919ef0b80f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:18.029271 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c953ffc-cbd0-4b4e-8f4c-6919ef0b80f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:18.037856 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae185e1-f50c-4f60-a512-16f15e4b3450 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:18.043251 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c953ffc-cbd0-4b4e-8f4c-6919ef0b80f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:29:18.054071 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-7c953ffc-cbd0-4b4e-8f4c-6919ef0b80f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:29:18.054071 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:18.054071 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:18.054071 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:18.054071 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:18.054071 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:18.054071 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:18.054071 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:18.054071 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:18.054071 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:18.054071 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:18.054071 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:18.054071 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:29:18.054071 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:29:18.054071 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:29:18.054071 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:29:18.055060 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:29:18.055060 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:29:18.055060 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:29:18.055060 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:29:18.055060 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:18.055060 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:18.055060 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:18.055060 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:18.055060 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:18.055060 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:18.055060 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:18.055060 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:18.055060 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:18.055060 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:18.055060 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:18.055060 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:18.055908 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:18.055908 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:18.055908 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:18.055908 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:18.055908 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:29:18.055908 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:29:18.055908 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:29:18.055908 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:29:18.055908 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:29:18.055908 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:29:18.055908 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:29:18.055908 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:18.058358 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 405/792] 10.222.0.36 () {56 vars in 1096 bytes} [Thu May 19 16:29:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:18.059397 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c953ffc-cbd0-4b4e-8f4c-6919ef0b80f0 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:18.075347 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887bdd6-d997-49e4-a57e-28bcc1b47276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:18.076120 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887bdd6-d997-49e4-a57e-28bcc1b47276 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:18.076477 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887bdd6-d997-49e4-a57e-28bcc1b47276 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:18.076828 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887bdd6-d997-49e4-a57e-28bcc1b47276 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:18.077775 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3887bdd6-d997-49e4-a57e-28bcc1b47276 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:18.078207 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3887bdd6-d997-49e4-a57e-28bcc1b47276 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:18.079858 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3887bdd6-d997-49e4-a57e-28bcc1b47276 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:18.099492 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 388/793] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:29:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:18.101455 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:18.102521 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887bdd6-d997-49e4-a57e-28bcc1b47276 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:18.117965 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:18.183757 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c953ffc-cbd0-4b4e-8f4c-6919ef0b80f0 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:18.226852 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887bdd6-d997-49e4-a57e-28bcc1b47276 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:18.228972 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360c0d7a-9db6-42fd-a597-bb3557bc8c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:18.230415 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c0d7a-9db6-42fd-a597-bb3557bc8c90 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:18.230415 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c0d7a-9db6-42fd-a597-bb3557bc8c90 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:18.230665 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c0d7a-9db6-42fd-a597-bb3557bc8c90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:18.243559 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360c0d7a-9db6-42fd-a597-bb3557bc8c90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:18.245342 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360c0d7a-9db6-42fd-a597-bb3557bc8c90 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:18.266308 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-360c0d7a-9db6-42fd-a597-bb3557bc8c90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:18.266308 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:18.266308 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:18.266308 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:18.266308 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:18.266308 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:18.266308 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:18.266308 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:18.266308 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:18.266308 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:18.266308 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:18.266308 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:18.266308 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:29:18.266308 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:29:18.266308 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:29:18.267273 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:29:18.267273 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:18.267273 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:18.267273 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:18.267273 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:18.267273 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:18.267273 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:18.267273 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:18.267273 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:18.267273 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:18.267273 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:18.267273 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:18.267273 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:18.267273 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:18.267273 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:18.267273 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:18.267273 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:18.268168 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:29:18.268168 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:29:18.268168 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:18.268168 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:18.272794 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 406/794] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:29:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:18.273935 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360c0d7a-9db6-42fd-a597-bb3557bc8c90 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:18.275195 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960020eb-3520-434a-9938-4e6d8d5e1eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:18.276172 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960020eb-3520-434a-9938-4e6d8d5e1eae None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:18.276824 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960020eb-3520-434a-9938-4e6d8d5e1eae None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:18.277361 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960020eb-3520-434a-9938-4e6d8d5e1eae None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:18.278488 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960020eb-3520-434a-9938-4e6d8d5e1eae None admin] RBAC: Adding query filter params (name=admin) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:18.279007 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960020eb-3520-434a-9938-4e6d8d5e1eae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:18.280722 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960020eb-3520-434a-9938-4e6d8d5e1eae None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:18.292090 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 389/795] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:29:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:18.293450 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960020eb-3520-434a-9938-4e6d8d5e1eae None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:18.369346 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:18.378517 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:18.391062 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360c0d7a-9db6-42fd-a597-bb3557bc8c90 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:18.414316 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960020eb-3520-434a-9938-4e6d8d5e1eae None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:18.429089 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d03ba4d-917a-45d8-a602-0da58bd73202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:18.429975 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d03ba4d-917a-45d8-a602-0da58bd73202 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:18.430602 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d03ba4d-917a-45d8-a602-0da58bd73202 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:18.430602 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d03ba4d-917a-45d8-a602-0da58bd73202 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:18.439410 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d03ba4d-917a-45d8-a602-0da58bd73202 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:18.441348 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d03ba4d-917a-45d8-a602-0da58bd73202 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:18.452231 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f33561-614d-4292-9f98-eb303cbca33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:18.453443 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f33561-614d-4292-9f98-eb303cbca33f None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:18.453739 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f33561-614d-4292-9f98-eb303cbca33f None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:18.454017 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f33561-614d-4292-9f98-eb303cbca33f None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:18.454754 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f33561-614d-4292-9f98-eb303cbca33f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:18.455065 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f33561-614d-4292-9f98-eb303cbca33f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:18.456438 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f33561-614d-4292-9f98-eb303cbca33f None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:18.457266 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-3d03ba4d-917a-45d8-a602-0da58bd73202 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 16:29:18.457266 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:18.457266 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:18.457266 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:18.457266 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:18.457266 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:18.457266 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:18.457266 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:18.457266 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:18.457266 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:18.457266 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:18.457266 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:18.457266 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 16:29:18.457266 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 16:29:18.457266 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 16:29:18.457266 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 16:29:18.458331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:18.458331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:18.458331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 16:29:18.458331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:29:18.458331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 16:29:18.458331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:29:18.458331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:18.458331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:18.458331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:18.458331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:18.458331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:18.458331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:18.458331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:18.458331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:18.458331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:18.458331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:18.458331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:18.458331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:18.459300 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:18.459300 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:18.459300 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 16:29:18.459300 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 16:29:18.459300 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 16:29:18.459300 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 16:29:18.459300 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 16:29:18.459300 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 16:29:18.459300 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 16:29:18.459300 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:18.459300 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 407/796] 10.222.0.36 () {56 vars in 1096 bytes} [Thu May 19 16:29:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:18.460501 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d03ba4d-917a-45d8-a602-0da58bd73202 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:18.481454 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 390/797] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:29:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:18.483208 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df2954-8136-4e2c-80ca-e5df6baf6c6f None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:18.483668 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df2954-8136-4e2c-80ca-e5df6baf6c6f None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:18.483969 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df2954-8136-4e2c-80ca-e5df6baf6c6f None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:18.485509 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 391/798] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:29:18.490225 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df2954-8136-4e2c-80ca-e5df6baf6c6f None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:18.578916 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d03ba4d-917a-45d8-a602-0da58bd73202 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:18.604493 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df2954-8136-4e2c-80ca-e5df6baf6c6f None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:18.618275 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e6014-bb02-48bb-b446-ecc7c9a1c2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:18.618981 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e6014-bb02-48bb-b446-ecc7c9a1c2ec None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:18.619275 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e6014-bb02-48bb-b446-ecc7c9a1c2ec None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:18.619539 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e6014-bb02-48bb-b446-ecc7c9a1c2ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:18.632756 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63e6014-bb02-48bb-b446-ecc7c9a1c2ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:18.634343 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63e6014-bb02-48bb-b446-ecc7c9a1c2ec None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:18.638752 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:18.642342 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:18.646424 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f534b2ad-a10e-4e31-b3b1-c2e399ea2c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:18.647121 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534b2ad-a10e-4e31-b3b1-c2e399ea2c9d None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:18.647402 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534b2ad-a10e-4e31-b3b1-c2e399ea2c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:18.647680 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534b2ad-a10e-4e31-b3b1-c2e399ea2c9d None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:18.648538 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f534b2ad-a10e-4e31-b3b1-c2e399ea2c9d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:18.648853 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f534b2ad-a10e-4e31-b3b1-c2e399ea2c9d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:18.650311 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f534b2ad-a10e-4e31-b3b1-c2e399ea2c9d None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:18.655351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-b63e6014-bb02-48bb-b446-ecc7c9a1c2ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:18.655351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:18.655351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:18.655351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:18.655351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:18.655351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:18.655351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:18.655351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:18.655351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:18.655351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:18.655351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:18.655351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:18.655351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:29:18.655351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:29:18.655351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:29:18.656918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:29:18.656918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:18.656918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:18.656918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:18.656918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:18.656918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:18.656918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:18.656918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:18.656918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:18.656918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:18.656918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:18.656918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:18.656918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:18.656918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:18.656918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:18.656918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:18.656918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:18.657857 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:29:18.657857 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:29:18.657857 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:18.657857 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:18.661791 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 408/799] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:29:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:18.662691 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e6014-bb02-48bb-b446-ecc7c9a1c2ec None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:18.670803 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 392/800] 10.222.0.36 () {56 vars in 1135 bytes} [Thu May 19 16:29:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:18.672662 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4facf8-7c20-4817-8366-e28df0a6a5f1 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:18.673006 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4facf8-7c20-4817-8366-e28df0a6a5f1 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:18.673312 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4facf8-7c20-4817-8366-e28df0a6a5f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:18.690941 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-3d4facf8-7c20-4817-8366-e28df0a6a5f1 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:18.692180 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-3d4facf8-7c20-4817-8366-e28df0a6a5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:18.767182 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 393/801] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:29:18.774317 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4facf8-7c20-4817-8366-e28df0a6a5f1 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:18.777765 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e6014-bb02-48bb-b446-ecc7c9a1c2ec None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:18.821247 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce06cb4b-165f-4148-b890-97d6f4211bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:18.822125 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06cb4b-165f-4148-b890-97d6f4211bcc None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:18.822357 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06cb4b-165f-4148-b890-97d6f4211bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:18.822630 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06cb4b-165f-4148-b890-97d6f4211bcc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:18.830881 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce06cb4b-165f-4148-b890-97d6f4211bcc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:18.832285 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce06cb4b-165f-4148-b890-97d6f4211bcc None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:18.847966 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-ce06cb4b-165f-4148-b890-97d6f4211bcc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 16:29:18.847966 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:18.847966 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:18.847966 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:18.847966 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:18.847966 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:18.847966 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:18.847966 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:18.847966 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:18.847966 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:18.847966 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:18.847966 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:18.847966 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 16:29:18.847966 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 16:29:18.847966 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 16:29:18.847966 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 16:29:18.848970 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:18.848970 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:18.848970 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 16:29:18.848970 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:29:18.848970 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 16:29:18.848970 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:29:18.848970 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:18.848970 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:18.848970 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:18.848970 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:18.848970 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:18.848970 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:18.848970 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:18.848970 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:18.848970 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:18.848970 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:18.848970 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:18.848970 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:18.849939 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:18.849939 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:18.849939 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 16:29:18.849939 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 16:29:18.849939 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 16:29:18.849939 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 16:29:18.849939 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 16:29:18.849939 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 16:29:18.849939 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 16:29:18.849939 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:18.849939 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 409/802] 10.222.0.36 () {56 vars in 1117 bytes} [Thu May 19 16:29:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:18.850547 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce06cb4b-165f-4148-b890-97d6f4211bcc None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:18.897855 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4facf8-7c20-4817-8366-e28df0a6a5f1 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:18.919589 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:18.923232 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:18.940141 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc9f40-8d12-4be2-8580-5c1bcfab4e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:18.940825 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc9f40-8d12-4be2-8580-5c1bcfab4e90 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:18.941112 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc9f40-8d12-4be2-8580-5c1bcfab4e90 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:18.941431 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc9f40-8d12-4be2-8580-5c1bcfab4e90 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:18.942171 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfc9f40-8d12-4be2-8580-5c1bcfab4e90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:18.942479 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfc9f40-8d12-4be2-8580-5c1bcfab4e90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:18.943859 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfc9f40-8d12-4be2-8580-5c1bcfab4e90 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:18.962761 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 394/803] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:29:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:18.962761 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc9f40-8d12-4be2-8580-5c1bcfab4e90 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:18.975145 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce06cb4b-165f-4148-b890-97d6f4211bcc None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:19.012530 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189e988b-5354-4382-bd49-81ba8578ed7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:19.013491 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e988b-5354-4382-bd49-81ba8578ed7e None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:19.013752 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e988b-5354-4382-bd49-81ba8578ed7e None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:19.014029 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e988b-5354-4382-bd49-81ba8578ed7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:19.026061 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189e988b-5354-4382-bd49-81ba8578ed7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:19.027538 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189e988b-5354-4382-bd49-81ba8578ed7e None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:19.047081 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-189e988b-5354-4382-bd49-81ba8578ed7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:19.047081 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:19.047081 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:19.047081 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:19.047081 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:19.047081 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:19.047081 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:19.047081 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:19.047081 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:19.047081 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:19.047081 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:19.047081 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:19.047081 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:29:19.047081 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:29:19.047081 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:29:19.053741 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:29:19.053741 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:19.053741 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:19.053741 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:19.053741 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:19.053741 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:19.053741 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:19.053741 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:19.053741 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:19.053741 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:19.053741 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:19.053741 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:19.053741 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:19.053741 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:19.053741 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:19.053741 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:19.053741 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:19.056927 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:29:19.056927 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:29:19.056927 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:19.056927 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:19.056927 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 410/804] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:29:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:19.056927 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189e988b-5354-4382-bd49-81ba8578ed7e None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:19.080160 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc9f40-8d12-4be2-8580-5c1bcfab4e90 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:19.115580 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe3b6c-4a73-4898-b904-030570d92c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:19.116302 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe3b6c-4a73-4898-b904-030570d92c5b None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:19.116585 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe3b6c-4a73-4898-b904-030570d92c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:19.116857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe3b6c-4a73-4898-b904-030570d92c5b None admin] PATH_INFO: `/v3/services/network` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:19.117544 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbe3b6c-4a73-4898-b904-030570d92c5b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:19.118940 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbe3b6c-4a73-4898-b904-030570d92c5b None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:19.135507 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-acbe3b6c-4a73-4898-b904-030570d92c5b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 16:29:19.135507 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:19.135507 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 16:29:19.135507 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 16:29:19.135507 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 16:29:19.135507 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 16:29:19.135507 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 16:29:19.135507 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:29:19.135507 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 16:29:19.135507 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:19.135507 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 16:29:19.135507 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:19.135507 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:19.135507 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:19.135507 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:19.135507 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:19.135507 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:19.137387 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:19.137387 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:19.137387 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:19.137387 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:19.137387 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:19.137387 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:19.137387 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 16:29:19.137387 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 16:29:19.137387 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 16:29:19.137387 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 16:29:19.137387 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:19.137387 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:19.137387 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:19.137387 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:19.137387 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:19.137387 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:19.138354 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:19.138354 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:19.138354 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:19.138354 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:19.138354 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:19.138354 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:19.138354 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:19.138354 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:19.138354 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:19.138354 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:19.138354 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 16:29:19.138354 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:29:19.138354 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 16:29:19.138354 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:19.141560 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 395/805] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:29:18 2022] GET /identity/v3/services/network => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:19.142786 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe3b6c-4a73-4898-b904-030570d92c5b None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:19.181418 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189e988b-5354-4382-bd49-81ba8578ed7e None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:19.192875 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:19.196321 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:19.221619 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516eff87-20fc-4b1d-9949-e1503cb45037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:19.222506 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516eff87-20fc-4b1d-9949-e1503cb45037 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:19.222797 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516eff87-20fc-4b1d-9949-e1503cb45037 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:19.223075 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516eff87-20fc-4b1d-9949-e1503cb45037 None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:19.224103 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516eff87-20fc-4b1d-9949-e1503cb45037 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:19.224471 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516eff87-20fc-4b1d-9949-e1503cb45037 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:19.225969 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516eff87-20fc-4b1d-9949-e1503cb45037 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:19.251946 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 411/806] 10.222.0.36 () {56 vars in 1149 bytes} [Thu May 19 16:29:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:19.251946 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516eff87-20fc-4b1d-9949-e1503cb45037 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:19.254959 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe3b6c-4a73-4898-b904-030570d92c5b None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:19.291495 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3247fd-214b-4012-b522-21c6d6dc36b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:19.292304 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3247fd-214b-4012-b522-21c6d6dc36b5 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:19.292594 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3247fd-214b-4012-b522-21c6d6dc36b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:19.292899 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3247fd-214b-4012-b522-21c6d6dc36b5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:19.301255 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3247fd-214b-4012-b522-21c6d6dc36b5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:19.302739 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3247fd-214b-4012-b522-21c6d6dc36b5 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:19.316991 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-3f3247fd-214b-4012-b522-21c6d6dc36b5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 16:29:19.316991 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:19.316991 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:19.316991 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:19.316991 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:19.316991 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:19.316991 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:19.316991 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:19.316991 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:19.316991 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:19.316991 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:19.316991 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:19.316991 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 16:29:19.316991 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 16:29:19.316991 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 16:29:19.316991 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 16:29:19.318163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:19.318163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:19.318163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 16:29:19.318163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:29:19.318163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 16:29:19.318163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:29:19.318163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:19.318163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:19.318163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:19.318163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:19.318163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:19.318163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:19.318163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:19.318163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:19.318163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:19.318163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:19.318163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:19.318163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:19.319096 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:19.319096 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:19.319096 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 16:29:19.319096 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 16:29:19.319096 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 16:29:19.319096 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 16:29:19.319096 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 16:29:19.319096 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 16:29:19.319096 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 16:29:19.319096 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:19.319096 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 396/807] 10.222.0.36 () {56 vars in 1093 bytes} [Thu May 19 16:29:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:19.319096 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3247fd-214b-4012-b522-21c6d6dc36b5 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:19.363980 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516eff87-20fc-4b1d-9949-e1503cb45037 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:19.399311 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951d21ff-6d27-4c28-b057-c7e4b086fe7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:19.400394 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d21ff-6d27-4c28-b057-c7e4b086fe7a None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:19.400686 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d21ff-6d27-4c28-b057-c7e4b086fe7a None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:19.400964 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d21ff-6d27-4c28-b057-c7e4b086fe7a None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:19.401734 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951d21ff-6d27-4c28-b057-c7e4b086fe7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:19.402398 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951d21ff-6d27-4c28-b057-c7e4b086fe7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:19.403786 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951d21ff-6d27-4c28-b057-c7e4b086fe7a None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:19.415249 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 412/808] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:29:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:19.416064 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951d21ff-6d27-4c28-b057-c7e4b086fe7a None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:19.424024 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3247fd-214b-4012-b522-21c6d6dc36b5 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:19.426388 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:19.445031 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:19.460561 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bcb405-5066-4014-b248-54f435fd25ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:19.461250 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bcb405-5066-4014-b248-54f435fd25ec None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:19.461536 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bcb405-5066-4014-b248-54f435fd25ec None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:19.461811 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bcb405-5066-4014-b248-54f435fd25ec None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:19.462531 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bcb405-5066-4014-b248-54f435fd25ec None admin] RBAC: Adding query filter params (name=network) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:19.462841 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bcb405-5066-4014-b248-54f435fd25ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:19.464325 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bcb405-5066-4014-b248-54f435fd25ec None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:19.468657 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 397/809] 10.222.0.36 () {56 vars in 1112 bytes} [Thu May 19 16:29:19 2022] GET /identity/v3/services?name=network => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:19.469476 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bcb405-5066-4014-b248-54f435fd25ec None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:19.530200 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951d21ff-6d27-4c28-b057-c7e4b086fe7a None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:19.574624 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ef51d1-e0cb-4a34-8cc1-985f92fa68a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:19.575423 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef51d1-e0cb-4a34-8cc1-985f92fa68a0 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:19.575706 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef51d1-e0cb-4a34-8cc1-985f92fa68a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:19.575984 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef51d1-e0cb-4a34-8cc1-985f92fa68a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:19.587822 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ef51d1-e0cb-4a34-8cc1-985f92fa68a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:19.589301 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ef51d1-e0cb-4a34-8cc1-985f92fa68a0 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:19.590123 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bcb405-5066-4014-b248-54f435fd25ec None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:19.606474 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-65ef51d1-e0cb-4a34-8cc1-985f92fa68a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:19.606474 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:19.606474 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:19.606474 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:19.606474 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:19.606474 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:19.606474 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:19.606474 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:19.606474 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:19.606474 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:19.606474 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:19.606474 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:19.606474 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:29:19.606474 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:29:19.606474 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:29:19.607455 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:29:19.607455 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:19.607455 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:19.607455 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:19.607455 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:19.607455 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:19.607455 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:19.607455 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:19.607455 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:19.607455 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:19.607455 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:19.607455 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:19.607455 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:19.607455 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:19.607455 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:19.607455 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:19.607455 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:19.608336 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:29:19.608336 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:29:19.608336 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:19.608336 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:19.608336 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 413/810] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:29:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:19.608968 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643200e4-0960-458b-85ca-3a32d4ce1446 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:19.609327 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643200e4-0960-458b-85ca-3a32d4ce1446 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:19.609604 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643200e4-0960-458b-85ca-3a32d4ce1446 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:19.611012 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 414/811] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:29:19.611560 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643200e4-0960-458b-85ca-3a32d4ce1446 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:19.629925 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d98919-1617-4f0c-82b8-666bfa821a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:19.630629 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d98919-1617-4f0c-82b8-666bfa821a97 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:19.630916 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d98919-1617-4f0c-82b8-666bfa821a97 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:19.631191 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d98919-1617-4f0c-82b8-666bfa821a97 None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:19.632044 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d98919-1617-4f0c-82b8-666bfa821a97 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:19.632352 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d98919-1617-4f0c-82b8-666bfa821a97 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:19.633820 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d98919-1617-4f0c-82b8-666bfa821a97 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:19.652287 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 398/812] 10.222.0.36 () {56 vars in 1133 bytes} [Thu May 19 16:29:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:19.652832 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d98919-1617-4f0c-82b8-666bfa821a97 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:19.680337 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:19.694534 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:19.715842 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643200e4-0960-458b-85ca-3a32d4ce1446 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:19.751212 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ef9f0-c178-4a01-a0ef-a364ea4ab782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:19.752011 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ef9f0-c178-4a01-a0ef-a364ea4ab782 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:19.752297 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ef9f0-c178-4a01-a0ef-a364ea4ab782 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:19.752584 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ef9f0-c178-4a01-a0ef-a364ea4ab782 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:19.755646 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d98919-1617-4f0c-82b8-666bfa821a97 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:19.760413 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04ef9f0-c178-4a01-a0ef-a364ea4ab782 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:19.761920 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04ef9f0-c178-4a01-a0ef-a364ea4ab782 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:19.776974 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-b04ef9f0-c178-4a01-a0ef-a364ea4ab782 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:29:19.776974 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:19.776974 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:19.776974 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:19.776974 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:19.776974 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:19.776974 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:19.776974 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:19.776974 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:19.776974 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:19.776974 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:19.776974 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:19.776974 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 16:29:19.776974 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 16:29:19.776974 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 16:29:19.778069 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 16:29:19.778069 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:19.778069 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:19.778069 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:19.778069 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:19.778069 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:19.778069 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:19.778069 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:19.778069 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:19.778069 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:19.778069 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:19.778069 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:19.778069 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:19.778069 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:19.778069 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:19.778069 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:19.778069 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:19.778964 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 16:29:19.778964 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 16:29:19.778964 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 16:29:19.778964 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 16:29:19.778964 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 16:29:19.778964 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 16:29:19.778964 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:29:19.778964 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:19.778964 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 415/813] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:29:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:19.779449 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ef9f0-c178-4a01-a0ef-a364ea4ab782 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:19.801434 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbfc6da-28bc-4c64-a4a6-7e85cf3220e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:19.802106 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbfc6da-28bc-4c64-a4a6-7e85cf3220e1 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:19.802390 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbfc6da-28bc-4c64-a4a6-7e85cf3220e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:19.802668 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbfc6da-28bc-4c64-a4a6-7e85cf3220e1 None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:19.803383 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbfc6da-28bc-4c64-a4a6-7e85cf3220e1 None admin] RBAC: Adding query filter params (type=network) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:19.803694 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbfc6da-28bc-4c64-a4a6-7e85cf3220e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:19.805069 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbfc6da-28bc-4c64-a4a6-7e85cf3220e1 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:19.809876 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 399/814] 10.222.0.36 () {56 vars in 1112 bytes} [Thu May 19 16:29:19 2022] GET /identity/v3/services?type=network => generated 350 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:19.811064 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36285f93-dbe9-4ed8-8cd0-c240ae1343c0 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:19.811355 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36285f93-dbe9-4ed8-8cd0-c240ae1343c0 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:19.811639 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36285f93-dbe9-4ed8-8cd0-c240ae1343c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:19.828128 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-36285f93-dbe9-4ed8-8cd0-c240ae1343c0 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:19.829373 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-36285f93-dbe9-4ed8-8cd0-c240ae1343c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:19.880783 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ef9f0-c178-4a01-a0ef-a364ea4ab782 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:19.906453 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 400/815] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:29:19.907241 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36285f93-dbe9-4ed8-8cd0-c240ae1343c0 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:19.915727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dd2dbd-dc35-4606-b409-2b61430a6f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:19.916398 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd2dbd-dc35-4606-b409-2b61430a6f13 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:19.916682 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd2dbd-dc35-4606-b409-2b61430a6f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:19.917014 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd2dbd-dc35-4606-b409-2b61430a6f13 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:19.917818 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dd2dbd-dc35-4606-b409-2b61430a6f13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:19.918135 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dd2dbd-dc35-4606-b409-2b61430a6f13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:19.919521 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dd2dbd-dc35-4606-b409-2b61430a6f13 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:19.929628 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:19.933093 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 416/816] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:29:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:19.934046 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dd2dbd-dc35-4606-b409-2b61430a6f13 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:19.943277 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:20.020876 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36285f93-dbe9-4ed8-8cd0-c240ae1343c0 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:20.037023 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dd2dbd-dc35-4606-b409-2b61430a6f13 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:20.056434 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782e1ad3-a810-4d3c-9e99-f8f9b39b6017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:20.057104 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e1ad3-a810-4d3c-9e99-f8f9b39b6017 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:20.057436 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e1ad3-a810-4d3c-9e99-f8f9b39b6017 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:20.057726 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e1ad3-a810-4d3c-9e99-f8f9b39b6017 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:20.072196 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd1a7a5-9f79-4c45-8103-d1cce8ad0ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:20.072862 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1a7a5-9f79-4c45-8103-d1cce8ad0ce8 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:20.073172 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1a7a5-9f79-4c45-8103-d1cce8ad0ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:20.073464 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1a7a5-9f79-4c45-8103-d1cce8ad0ce8 None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:20.074263 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd1a7a5-9f79-4c45-8103-d1cce8ad0ce8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:20.074577 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd1a7a5-9f79-4c45-8103-d1cce8ad0ce8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:20.076038 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd1a7a5-9f79-4c45-8103-d1cce8ad0ce8 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:20.088230 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 417/817] 10.222.0.36 () {56 vars in 1148 bytes} [Thu May 19 16:29:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:20.089022 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd1a7a5-9f79-4c45-8103-d1cce8ad0ce8 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:20.133430 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782e1ad3-a810-4d3c-9e99-f8f9b39b6017 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:20.134891 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782e1ad3-a810-4d3c-9e99-f8f9b39b6017 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:20.149463 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:20.156752 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:20.193322 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd1a7a5-9f79-4c45-8103-d1cce8ad0ce8 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:20.226557 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1148d37e-c676-458a-9bed-5c0b3cbc1ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:20.227215 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1148d37e-c676-458a-9bed-5c0b3cbc1ca0 None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:20.227499 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1148d37e-c676-458a-9bed-5c0b3cbc1ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:20.227777 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1148d37e-c676-458a-9bed-5c0b3cbc1ca0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:20.228514 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1148d37e-c676-458a-9bed-5c0b3cbc1ca0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:20.229930 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1148d37e-c676-458a-9bed-5c0b3cbc1ca0 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:20.238263 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 401/818] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:29:19 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 16:29:20.239116 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782e1ad3-a810-4d3c-9e99-f8f9b39b6017 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:20.251414 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 418/819] 10.222.0.36 () {60 vars in 1144 bytes} [Thu May 19 16:29:20 2022] POST /identity/v3/endpoints => generated 327 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:29:20.252252 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1148d37e-c676-458a-9bed-5c0b3cbc1ca0 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:20.363246 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782e1ad3-a810-4d3c-9e99-f8f9b39b6017 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:20.388378 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:20.392070 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1148d37e-c676-458a-9bed-5c0b3cbc1ca0 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:20.407904 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:20.410571 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee06da6-7ce7-4286-bff2-d4c05d542b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:20.411458 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee06da6-7ce7-4286-bff2-d4c05d542b49 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:20.411674 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee06da6-7ce7-4286-bff2-d4c05d542b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:20.411953 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee06da6-7ce7-4286-bff2-d4c05d542b49 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:20.412632 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee06da6-7ce7-4286-bff2-d4c05d542b49 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:20.414172 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee06da6-7ce7-4286-bff2-d4c05d542b49 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:20.427655 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-5ee06da6-7ce7-4286-bff2-d4c05d542b49 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 16:29:20.427655 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:20.427655 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 16:29:20.427655 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 16:29:20.427655 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 16:29:20.427655 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 16:29:20.427655 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 16:29:20.427655 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:29:20.427655 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 16:29:20.427655 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:20.427655 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 16:29:20.427655 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:20.427655 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:20.427655 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:20.427655 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:20.427655 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:20.428940 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:20.428940 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:20.428940 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:20.428940 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:20.428940 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:20.428940 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:20.428940 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:20.428940 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 16:29:20.428940 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 16:29:20.428940 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 16:29:20.428940 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 16:29:20.428940 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:20.428940 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:20.428940 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:20.428940 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:20.428940 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:20.429858 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:20.429858 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:20.429858 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:20.429858 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:20.429858 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:20.429858 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:20.429858 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:20.429858 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:20.429858 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:20.429858 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:20.429858 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:20.429858 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 16:29:20.429858 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:29:20.429858 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 16:29:20.429858 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:20.429858 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 402/820] 10.222.0.36 () {56 vars in 1129 bytes} [Thu May 19 16:29:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:20.430717 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee06da6-7ce7-4286-bff2-d4c05d542b49 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:20.441401 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89be8edc-005f-4f1b-946b-3cdbc9264ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:20.442224 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be8edc-005f-4f1b-946b-3cdbc9264ceb None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:20.442515 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be8edc-005f-4f1b-946b-3cdbc9264ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:20.443682 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be8edc-005f-4f1b-946b-3cdbc9264ceb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:20.452783 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89be8edc-005f-4f1b-946b-3cdbc9264ceb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:20.454333 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89be8edc-005f-4f1b-946b-3cdbc9264ceb None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:20.471253 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-89be8edc-005f-4f1b-946b-3cdbc9264ceb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:29:20.471253 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:20.471253 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:20.471253 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:20.471253 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:20.471253 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:20.471253 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:20.471253 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:20.471253 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:20.471253 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:20.471253 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:20.471253 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:20.471253 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 16:29:20.471253 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 16:29:20.471253 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 16:29:20.472728 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 16:29:20.472728 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:20.472728 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:20.472728 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:20.472728 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:20.472728 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:20.472728 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:20.472728 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:20.472728 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:20.472728 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:20.472728 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:20.472728 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:20.472728 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:20.472728 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:20.472728 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:20.472728 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:20.472728 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:20.474519 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 16:29:20.474519 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 16:29:20.474519 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 16:29:20.474519 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 16:29:20.474519 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 16:29:20.474519 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 16:29:20.474519 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:29:20.474519 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:20.474519 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 419/821] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:29:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:20.474519 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89be8edc-005f-4f1b-946b-3cdbc9264ceb None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:20.563579 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee06da6-7ce7-4286-bff2-d4c05d542b49 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:20.588504 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89be8edc-005f-4f1b-946b-3cdbc9264ceb None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:20.598075 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8584fb7-c6e4-45f1-a417-9df65e8e5a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:20.599067 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8584fb7-c6e4-45f1-a417-9df65e8e5a17 None admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:20.599433 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8584fb7-c6e4-45f1-a417-9df65e8e5a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:20.599739 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8584fb7-c6e4-45f1-a417-9df65e8e5a17 None admin] PATH_INFO: `/v3/projects/631c76794a124d6da22c626e590e4352/users/ffe7cebc6f3b40d8a8e1866ed2da9822/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:20.602089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8584fb7-c6e4-45f1-a417-9df65e8e5a17 None admin] RBAC: Authorizing `identity:create_grant(project_id=631c76794a124d6da22c626e590e4352, user_id=ffe7cebc6f3b40d8a8e1866ed2da9822, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:20.603564 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8584fb7-c6e4-45f1-a417-9df65e8e5a17 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:20.616084 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 403/822] 10.222.0.36 () {58 vars in 1439 bytes} [Thu May 19 16:29:20 2022] PUT /identity/v3/projects/631c76794a124d6da22c626e590e4352/users/ffe7cebc6f3b40d8a8e1866ed2da9822/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:29:20.616901 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8584fb7-c6e4-45f1-a417-9df65e8e5a17 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:20.623929 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f722ff8-3221-4393-bfcb-f2e5166e714d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:20.624763 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f722ff8-3221-4393-bfcb-f2e5166e714d None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:20.625049 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f722ff8-3221-4393-bfcb-f2e5166e714d None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:20.625378 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f722ff8-3221-4393-bfcb-f2e5166e714d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:20.637552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f722ff8-3221-4393-bfcb-f2e5166e714d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:20.639107 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f722ff8-3221-4393-bfcb-f2e5166e714d None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:20.662306 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-4f722ff8-3221-4393-bfcb-f2e5166e714d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:20.662306 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:20.662306 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:20.662306 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:20.662306 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:20.662306 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:20.662306 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:20.662306 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:20.662306 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:20.662306 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:20.662306 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:20.662306 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:20.662306 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:29:20.662306 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:29:20.662306 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:29:20.663518 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:29:20.663518 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:20.663518 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:20.663518 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:20.663518 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:20.663518 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:20.663518 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:20.663518 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:20.663518 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:20.663518 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:20.663518 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:20.663518 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:20.663518 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:20.663518 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:20.663518 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:20.663518 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:20.663518 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:20.664490 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:29:20.664490 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:29:20.664490 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:20.664490 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:20.664490 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 420/823] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:29:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:20.668995 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f722ff8-3221-4393-bfcb-f2e5166e714d None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:20.688867 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:20.700006 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:20.752735 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8584fb7-c6e4-45f1-a417-9df65e8e5a17 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:20.790193 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f722ff8-3221-4393-bfcb-f2e5166e714d None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:20.792538 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90082b51-4f82-49f5-9876-1b3df19b6cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:20.793260 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90082b51-4f82-49f5-9876-1b3df19b6cb4 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:20.793549 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90082b51-4f82-49f5-9876-1b3df19b6cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:20.793827 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90082b51-4f82-49f5-9876-1b3df19b6cb4 None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:20.794547 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90082b51-4f82-49f5-9876-1b3df19b6cb4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:20.794858 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90082b51-4f82-49f5-9876-1b3df19b6cb4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:20.796256 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90082b51-4f82-49f5-9876-1b3df19b6cb4 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:20.802185 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 404/824] 10.222.0.36 () {56 vars in 1124 bytes} [Thu May 19 16:29:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:20.802521 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90082b51-4f82-49f5-9876-1b3df19b6cb4 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:20.851012 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a2321-8ccf-4f28-86f5-9476124caa46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:20.852068 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a2321-8ccf-4f28-86f5-9476124caa46 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:20.852479 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a2321-8ccf-4f28-86f5-9476124caa46 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:20.852940 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a2321-8ccf-4f28-86f5-9476124caa46 None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:20.853969 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94a2321-8ccf-4f28-86f5-9476124caa46 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:20.854447 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94a2321-8ccf-4f28-86f5-9476124caa46 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:20.856208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94a2321-8ccf-4f28-86f5-9476124caa46 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:20.871124 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 421/825] 10.222.0.36 () {56 vars in 1148 bytes} [Thu May 19 16:29:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:20.871593 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a2321-8ccf-4f28-86f5-9476124caa46 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:20.934721 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90082b51-4f82-49f5-9876-1b3df19b6cb4 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:20.944924 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:20.972128 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2991009-7103-4c53-a5fb-2e92ee529b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:20.973583 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2991009-7103-4c53-a5fb-2e92ee529b60 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:20.974470 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2991009-7103-4c53-a5fb-2e92ee529b60 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:20.975076 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2991009-7103-4c53-a5fb-2e92ee529b60 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:20.976090 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2991009-7103-4c53-a5fb-2e92ee529b60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:20.976410 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2991009-7103-4c53-a5fb-2e92ee529b60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:20.977862 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2991009-7103-4c53-a5fb-2e92ee529b60 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:20.980678 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:20.990814 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 405/826] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:29:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:20.991624 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2991009-7103-4c53-a5fb-2e92ee529b60 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:21.002867 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a2321-8ccf-4f28-86f5-9476124caa46 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:21.042160 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c4211-3b51-43de-9f04-5c820ac874ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:21.042835 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c4211-3b51-43de-9f04-5c820ac874ad None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:21.043123 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c4211-3b51-43de-9f04-5c820ac874ad None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:21.043464 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c4211-3b51-43de-9f04-5c820ac874ad None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:21.044190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c4211-3b51-43de-9f04-5c820ac874ad None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:21.044503 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c4211-3b51-43de-9f04-5c820ac874ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:21.046015 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c4211-3b51-43de-9f04-5c820ac874ad None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:21.051723 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 422/827] 10.222.0.36 () {56 vars in 1124 bytes} [Thu May 19 16:29:20 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:21.052362 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c4211-3b51-43de-9f04-5c820ac874ad None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:21.111363 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2991009-7103-4c53-a5fb-2e92ee529b60 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:21.151400 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df99131-48dd-4dc7-9354-96e2e4dcabfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:21.152299 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df99131-48dd-4dc7-9354-96e2e4dcabfc None admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:21.152612 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df99131-48dd-4dc7-9354-96e2e4dcabfc None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:21.152896 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df99131-48dd-4dc7-9354-96e2e4dcabfc None admin] PATH_INFO: `/v3/projects/631c76794a124d6da22c626e590e4352/users/a30dc1dd2b00463bb3c7957864c56043/roles/727026b648e1429ca3697e04800b328a` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:21.169068 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df99131-48dd-4dc7-9354-96e2e4dcabfc None admin] RBAC: Authorizing `identity:create_grant(project_id=631c76794a124d6da22c626e590e4352, user_id=a30dc1dd2b00463bb3c7957864c56043, role_id=727026b648e1429ca3697e04800b328a)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:21.170561 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df99131-48dd-4dc7-9354-96e2e4dcabfc None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:21.175346 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c4211-3b51-43de-9f04-5c820ac874ad None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:21.184997 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 406/828] 10.222.0.36 () {58 vars in 1439 bytes} [Thu May 19 16:29:20 2022] PUT /identity/v3/projects/631c76794a124d6da22c626e590e4352/users/a30dc1dd2b00463bb3c7957864c56043/roles/727026b648e1429ca3697e04800b328a => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:29:21.186027 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df99131-48dd-4dc7-9354-96e2e4dcabfc None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:21.215584 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2fdbdd-1965-4cf2-ac46-550e77a027d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:21.216468 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2fdbdd-1965-4cf2-ac46-550e77a027d2 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:21.216766 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2fdbdd-1965-4cf2-ac46-550e77a027d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:21.217051 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2fdbdd-1965-4cf2-ac46-550e77a027d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:21.220117 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:21.227712 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2fdbdd-1965-4cf2-ac46-550e77a027d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:21.229284 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2fdbdd-1965-4cf2-ac46-550e77a027d2 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:21.247335 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-4c2fdbdd-1965-4cf2-ac46-550e77a027d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:29:21.247335 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:21.247335 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:21.247335 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:21.247335 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:21.247335 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:21.247335 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:21.247335 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:21.247335 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:21.247335 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:21.247335 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:21.247335 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:21.247335 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 16:29:21.247335 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 16:29:21.247335 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 16:29:21.248362 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 16:29:21.248362 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:21.248362 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:21.248362 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:21.248362 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:21.248362 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:21.248362 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:21.248362 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:21.248362 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:21.248362 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:21.248362 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:21.248362 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:21.248362 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:21.248362 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:21.248362 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:21.248362 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:21.248362 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:21.249439 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 16:29:21.249439 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 16:29:21.249439 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 16:29:21.249439 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 16:29:21.249439 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 16:29:21.249439 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 16:29:21.249439 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:29:21.249439 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:21.253603 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 423/829] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:29:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:21.255568 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2fdbdd-1965-4cf2-ac46-550e77a027d2 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:21.269395 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:21.347261 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df99131-48dd-4dc7-9354-96e2e4dcabfc None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:21.390103 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2fdbdd-1965-4cf2-ac46-550e77a027d2 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:21.400774 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2007c876-3529-40da-9ed9-13630658f5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:21.401506 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2007c876-3529-40da-9ed9-13630658f5f0 None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:21.401802 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2007c876-3529-40da-9ed9-13630658f5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:21.402089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2007c876-3529-40da-9ed9-13630658f5f0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:21.402857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2007c876-3529-40da-9ed9-13630658f5f0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:21.405707 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2007c876-3529-40da-9ed9-13630658f5f0 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:21.430406 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 407/830] 10.222.0.36 () {60 vars in 1144 bytes} [Thu May 19 16:29:21 2022] POST /identity/v3/endpoints => generated 346 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:29:21.446120 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde67086-8859-4aa9-8a0d-4b08ccbc4c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:21.447571 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde67086-8859-4aa9-8a0d-4b08ccbc4c7f None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:21.448077 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde67086-8859-4aa9-8a0d-4b08ccbc4c7f None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:21.448586 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde67086-8859-4aa9-8a0d-4b08ccbc4c7f None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:21.449746 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde67086-8859-4aa9-8a0d-4b08ccbc4c7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:21.450293 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde67086-8859-4aa9-8a0d-4b08ccbc4c7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:21.452195 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde67086-8859-4aa9-8a0d-4b08ccbc4c7f None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:21.467829 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 424/831] 10.222.0.36 () {56 vars in 1148 bytes} [Thu May 19 16:29:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:21.473935 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2007c876-3529-40da-9ed9-13630658f5f0 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:21.567778 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:21.598976 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2007c876-3529-40da-9ed9-13630658f5f0 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:21.636615 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c4d322-0103-4a8a-97d0-e1e1e4ec9244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:21.637313 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c4d322-0103-4a8a-97d0-e1e1e4ec9244 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:21.637598 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c4d322-0103-4a8a-97d0-e1e1e4ec9244 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:21.637876 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c4d322-0103-4a8a-97d0-e1e1e4ec9244 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:21.638722 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c4d322-0103-4a8a-97d0-e1e1e4ec9244 None admin] RBAC: Adding query filter params (role.id=e2915fe109244a07a9be96bd2b219fdd, scope.project.id=631c76794a124d6da22c626e590e4352, user.id=58bdeada00b74b7b8e186e6dc51e6dfb) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:21.639039 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c4d322-0103-4a8a-97d0-e1e1e4ec9244 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:21.640506 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c4d322-0103-4a8a-97d0-e1e1e4ec9244 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:21.647828 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 408/832] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:29:21 2022] GET /identity/v3/role_assignments?user.id=58bdeada00b74b7b8e186e6dc51e6dfb&scope.project.id=631c76794a124d6da22c626e590e4352&role.id=e2915fe109244a07a9be96bd2b219fdd => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:22.923614 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128f872-fec3-4b9b-a0cc-ab6281801276 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:22.924148 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128f872-fec3-4b9b-a0cc-ab6281801276 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:22.924237 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128f872-fec3-4b9b-a0cc-ab6281801276 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:22.925906 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 425/833] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:29:22.931847 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611bab5-de39-4d99-876a-3cdcbb689c20 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:22.932308 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611bab5-de39-4d99-876a-3cdcbb689c20 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:22.932521 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611bab5-de39-4d99-876a-3cdcbb689c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:22.950475 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-a611bab5-de39-4d99-876a-3cdcbb689c20 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:22.951676 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-a611bab5-de39-4d99-876a-3cdcbb689c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:23.025285 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 409/834] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:22 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:29:23.045062 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d128f872-fec3-4b9b-a0cc-ab6281801276 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:23.114816 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:23.162035 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d128f872-fec3-4b9b-a0cc-ab6281801276 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:23.208589 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee06b1-4c31-4f66-9813-fc39292f295b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:23.210332 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee06b1-4c31-4f66-9813-fc39292f295b None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:23.210826 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee06b1-4c31-4f66-9813-fc39292f295b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:23.211560 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee06b1-4c31-4f66-9813-fc39292f295b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:23.226353 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ee06b1-4c31-4f66-9813-fc39292f295b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:29:23.236533 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-a7ee06b1-4c31-4f66-9813-fc39292f295b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:29:23.236533 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:23.236533 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:23.236533 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:23.236533 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:23.236533 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:23.236533 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:23.236533 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:23.236533 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:23.236533 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:23.236533 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:23.236533 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:23.236533 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:29:23.236533 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:29:23.236533 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:29:23.236533 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:29:23.238095 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:29:23.238095 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:29:23.238095 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:29:23.238095 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:29:23.238095 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:23.238095 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:23.238095 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:23.238095 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:23.238095 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:23.238095 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:23.238095 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:23.238095 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:23.238095 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:23.238095 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:23.238095 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:23.238095 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:23.238929 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:23.238929 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:23.238929 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:23.238929 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:23.238929 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:29:23.238929 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:29:23.238929 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:29:23.238929 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:29:23.238929 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:29:23.238929 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:29:23.238929 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:29:23.238929 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:23.241742 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 426/835] 10.222.0.36 () {56 vars in 1096 bytes} [Thu May 19 16:29:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:23.250270 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a611bab5-de39-4d99-876a-3cdcbb689c20 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:23.323339 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:23.363188 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a611bab5-de39-4d99-876a-3cdcbb689c20 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:23.399999 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6a23c0-a5a7-410f-a786-adb93975141e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:23.401133 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a23c0-a5a7-410f-a786-adb93975141e None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:23.401656 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a23c0-a5a7-410f-a786-adb93975141e None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:23.402139 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a23c0-a5a7-410f-a786-adb93975141e None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:23.403104 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6a23c0-a5a7-410f-a786-adb93975141e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:23.403623 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6a23c0-a5a7-410f-a786-adb93975141e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:23.405239 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6a23c0-a5a7-410f-a786-adb93975141e None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:23.415466 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 410/836] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:29:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:23.422891 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee06b1-4c31-4f66-9813-fc39292f295b None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:23.538684 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee06b1-4c31-4f66-9813-fc39292f295b None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:23.570056 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:23.584095 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246e0acb-9ed5-444a-8612-9528cb9f1322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:23.586037 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e0acb-9ed5-444a-8612-9528cb9f1322 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:23.586905 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e0acb-9ed5-444a-8612-9528cb9f1322 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:23.587743 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e0acb-9ed5-444a-8612-9528cb9f1322 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:23.606422 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246e0acb-9ed5-444a-8612-9528cb9f1322 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:23.608281 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246e0acb-9ed5-444a-8612-9528cb9f1322 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:23.633004 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-246e0acb-9ed5-444a-8612-9528cb9f1322 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:23.633004 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:23.633004 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:23.633004 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:23.633004 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:23.633004 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:23.633004 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:23.633004 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:23.633004 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:23.633004 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:23.633004 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:23.633004 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:23.633004 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:29:23.633004 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:29:23.633004 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:29:23.633984 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:29:23.633984 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:23.633984 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:23.633984 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:23.633984 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:23.633984 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:23.633984 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:23.633984 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:23.633984 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:23.633984 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:23.633984 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:23.633984 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:23.633984 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:23.633984 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:23.633984 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:23.633984 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:23.633984 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:23.634875 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:29:23.634875 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:29:23.634875 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:23.634875 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:23.637187 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 427/837] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:29:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:23.646307 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6a23c0-a5a7-410f-a786-adb93975141e None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:23.654429 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d1525-07c1-464b-a10d-9996c63c2385 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:23.654734 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d1525-07c1-464b-a10d-9996c63c2385 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:23.654991 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d1525-07c1-464b-a10d-9996c63c2385 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:23.657240 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 428/838] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 16:29:23.662453 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97597d2-76dc-47ee-8f3f-3cdc0d3bad6b None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:23.662742 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97597d2-76dc-47ee-8f3f-3cdc0d3bad6b None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:23.663020 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97597d2-76dc-47ee-8f3f-3cdc0d3bad6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:23.682716 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-b97597d2-76dc-47ee-8f3f-3cdc0d3bad6b None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:23.683996 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b97597d2-76dc-47ee-8f3f-3cdc0d3bad6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:23.758802 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6a23c0-a5a7-410f-a786-adb93975141e None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:23.764761 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 429/839] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:23 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 16:29:23.766873 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3b73b-ebf0-48ed-add5-8e45dd4e1d0d None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:23.767119 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3b73b-ebf0-48ed-add5-8e45dd4e1d0d None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:23.767409 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3b73b-ebf0-48ed-add5-8e45dd4e1d0d None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:23.769038 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 430/840] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:29:23.774454 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c5e0f-99fa-42cf-b4b6-4f9328a20c25 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:23.774744 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c5e0f-99fa-42cf-b4b6-4f9328a20c25 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:23.775024 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c5e0f-99fa-42cf-b4b6-4f9328a20c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:23.792633 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:23.794688 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-5d7c5e0f-99fa-42cf-b4b6-4f9328a20c25 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:23.796110 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-5d7c5e0f-99fa-42cf-b4b6-4f9328a20c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:23.800274 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd96a35c-f105-4f1f-98f7-663df6cc06b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:23.801194 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd96a35c-f105-4f1f-98f7-663df6cc06b3 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:23.801668 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd96a35c-f105-4f1f-98f7-663df6cc06b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:23.802147 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd96a35c-f105-4f1f-98f7-663df6cc06b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:23.803089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd96a35c-f105-4f1f-98f7-663df6cc06b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:23.803584 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd96a35c-f105-4f1f-98f7-663df6cc06b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:23.805142 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd96a35c-f105-4f1f-98f7-663df6cc06b3 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:23.817897 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 411/841] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:29:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:23.819144 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd96a35c-f105-4f1f-98f7-663df6cc06b3 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:23.881927 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 431/842] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:23 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 16:29:23.882617 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7c5e0f-99fa-42cf-b4b6-4f9328a20c25 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:23.939460 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:23.949556 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd96a35c-f105-4f1f-98f7-663df6cc06b3 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:23.986687 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc1c2d-1fe9-426b-8eae-45d01c94c14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:23.987525 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc1c2d-1fe9-426b-8eae-45d01c94c14d None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:23.987815 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc1c2d-1fe9-426b-8eae-45d01c94c14d None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:23.988094 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc1c2d-1fe9-426b-8eae-45d01c94c14d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:23.998722 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bc1c2d-1fe9-426b-8eae-45d01c94c14d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:29:24.007119 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-d7bc1c2d-1fe9-426b-8eae-45d01c94c14d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 16:29:24.007119 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:24.007119 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:24.007119 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:24.007119 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:24.007119 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:24.007119 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:24.007119 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:24.007119 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:24.007119 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:24.007119 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:24.007119 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:24.007119 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:29:24.007119 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:29:24.007119 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:29:24.007119 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:29:24.008168 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:29:24.008168 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:29:24.008168 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:29:24.008168 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:29:24.008168 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:24.008168 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:24.008168 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:24.008168 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:24.008168 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:24.008168 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:24.008168 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:24.008168 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:24.008168 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:24.008168 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:24.008168 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:24.008168 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:24.009027 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:24.009027 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:24.009027 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:24.009027 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:24.009027 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:29:24.009027 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:29:24.009027 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:29:24.009027 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:29:24.009027 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:29:24.009027 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:29:24.009027 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 16:29:24.009027 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:24.013432 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 412/843] 10.222.0.36 () {56 vars in 1102 bytes} [Thu May 19 16:29:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:24.014157 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc1c2d-1fe9-426b-8eae-45d01c94c14d None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:24.019264 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7c5e0f-99fa-42cf-b4b6-4f9328a20c25 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:24.055947 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab285110-9550-4030-b0ff-eb6e87b21819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:24.056751 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab285110-9550-4030-b0ff-eb6e87b21819 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:24.057034 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab285110-9550-4030-b0ff-eb6e87b21819 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:24.057341 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab285110-9550-4030-b0ff-eb6e87b21819 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:24.065100 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab285110-9550-4030-b0ff-eb6e87b21819 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:24.066582 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab285110-9550-4030-b0ff-eb6e87b21819 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:24.072803 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:24.081769 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-ab285110-9550-4030-b0ff-eb6e87b21819 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 16:29:24.081769 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:24.081769 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:24.081769 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:24.081769 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:24.081769 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:24.081769 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:24.081769 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:24.081769 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:24.081769 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:24.081769 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:24.081769 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:24.081769 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 16:29:24.081769 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 16:29:24.081769 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 16:29:24.081769 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 16:29:24.082736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:24.082736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:24.082736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 16:29:24.082736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:29:24.082736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 16:29:24.082736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:29:24.082736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:24.082736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:24.082736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:24.082736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:24.082736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:24.082736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:24.082736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:24.082736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:24.082736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:24.082736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:24.082736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:24.082736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:24.083696 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:24.083696 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:24.083696 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 16:29:24.083696 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 16:29:24.083696 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 16:29:24.083696 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 16:29:24.083696 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 16:29:24.083696 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 16:29:24.083696 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 16:29:24.083696 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:24.083696 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 432/844] 10.222.0.36 () {56 vars in 1096 bytes} [Thu May 19 16:29:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:24.084310 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab285110-9550-4030-b0ff-eb6e87b21819 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:24.129534 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc1c2d-1fe9-426b-8eae-45d01c94c14d None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:24.165142 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be427d2-eda7-4218-8d7c-fa307bb2843f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:24.165972 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be427d2-eda7-4218-8d7c-fa307bb2843f None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:24.166258 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be427d2-eda7-4218-8d7c-fa307bb2843f None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:24.166520 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be427d2-eda7-4218-8d7c-fa307bb2843f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:24.167182 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be427d2-eda7-4218-8d7c-fa307bb2843f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:24.168555 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be427d2-eda7-4218-8d7c-fa307bb2843f None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:24.178652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:24.180526 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-6be427d2-eda7-4218-8d7c-fa307bb2843f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 16:29:24.180526 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:24.180526 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 16:29:24.180526 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 16:29:24.180526 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 16:29:24.180526 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 16:29:24.180526 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 16:29:24.180526 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:29:24.180526 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 16:29:24.180526 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:24.180526 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 16:29:24.180526 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:24.180526 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:24.180526 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:24.180526 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:24.180526 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:24.180526 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:24.181644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:24.181644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:24.181644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:24.181644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:24.181644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:24.181644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:24.181644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 16:29:24.181644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 16:29:24.181644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 16:29:24.181644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 16:29:24.181644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:24.181644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:24.181644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:24.181644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:24.181644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:24.181644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:24.182523 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:24.182523 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:24.182523 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:24.182523 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:24.182523 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:24.182523 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:24.182523 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:24.182523 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:24.182523 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:24.182523 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:24.182523 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 16:29:24.182523 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:29:24.182523 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 16:29:24.182523 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:24.182523 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 413/845] 10.222.0.36 () {56 vars in 1114 bytes} [Thu May 19 16:29:24 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:24.183351 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e80a31-ca1c-4e11-acea-2224dd139a08 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:24.183645 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e80a31-ca1c-4e11-acea-2224dd139a08 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:24.183899 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e80a31-ca1c-4e11-acea-2224dd139a08 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:24.185147 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 414/846] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:29:24.185861 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e80a31-ca1c-4e11-acea-2224dd139a08 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:24.191330 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab285110-9550-4030-b0ff-eb6e87b21819 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:24.226025 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a325e09-ccb1-48a4-8674-8246cac19202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:24.226690 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a325e09-ccb1-48a4-8674-8246cac19202 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:24.226972 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a325e09-ccb1-48a4-8674-8246cac19202 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:24.227245 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a325e09-ccb1-48a4-8674-8246cac19202 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:24.227992 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a325e09-ccb1-48a4-8674-8246cac19202 None admin] RBAC: Adding query filter params (name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:24.228310 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a325e09-ccb1-48a4-8674-8246cac19202 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:24.229714 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a325e09-ccb1-48a4-8674-8246cac19202 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:24.240111 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 433/847] 10.222.0.36 () {56 vars in 1103 bytes} [Thu May 19 16:29:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:24.240111 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a325e09-ccb1-48a4-8674-8246cac19202 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:24.288593 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e80a31-ca1c-4e11-acea-2224dd139a08 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:24.302621 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:24.323405 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beed89e-5009-42f3-94a2-2170cf384ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:24.324087 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beed89e-5009-42f3-94a2-2170cf384ca3 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:24.324376 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beed89e-5009-42f3-94a2-2170cf384ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:24.324655 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beed89e-5009-42f3-94a2-2170cf384ca3 None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:24.325554 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beed89e-5009-42f3-94a2-2170cf384ca3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:24.325872 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beed89e-5009-42f3-94a2-2170cf384ca3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:24.327322 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beed89e-5009-42f3-94a2-2170cf384ca3 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:24.339065 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a325e09-ccb1-48a4-8674-8246cac19202 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:24.344459 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 415/848] 10.222.0.36 () {56 vars in 1135 bytes} [Thu May 19 16:29:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:24.345815 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eea2119-4be9-493d-b7f0-85e9c3e9381c None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:24.346107 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eea2119-4be9-493d-b7f0-85e9c3e9381c None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:24.346387 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eea2119-4be9-493d-b7f0-85e9c3e9381c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:24.363637 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-6eea2119-4be9-493d-b7f0-85e9c3e9381c None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:24.364849 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-6eea2119-4be9-493d-b7f0-85e9c3e9381c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:24.374398 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b45796-073a-47fd-a9c2-83222440c0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:24.375616 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b45796-073a-47fd-a9c2-83222440c0c2 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:24.375900 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b45796-073a-47fd-a9c2-83222440c0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:24.376176 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b45796-073a-47fd-a9c2-83222440c0c2 None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:24.376884 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b45796-073a-47fd-a9c2-83222440c0c2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:24.377223 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b45796-073a-47fd-a9c2-83222440c0c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:24.378596 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b45796-073a-47fd-a9c2-83222440c0c2 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:24.383087 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 434/849] 10.222.0.36 () {56 vars in 1114 bytes} [Thu May 19 16:29:24 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:24.383863 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b45796-073a-47fd-a9c2-83222440c0c2 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:24.409443 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:24.432546 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 416/850] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:24 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:29:24.433434 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eea2119-4be9-493d-b7f0-85e9c3e9381c None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:24.486241 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b45796-073a-47fd-a9c2-83222440c0c2 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:24.520730 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c0071a-3dbe-4c80-98e6-1d96ae1d0e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:24.521654 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0071a-3dbe-4c80-98e6-1d96ae1d0e9d None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:24.522004 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0071a-3dbe-4c80-98e6-1d96ae1d0e9d None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:24.522283 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0071a-3dbe-4c80-98e6-1d96ae1d0e9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:24.532805 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c0071a-3dbe-4c80-98e6-1d96ae1d0e9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:24.534293 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c0071a-3dbe-4c80-98e6-1d96ae1d0e9d None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:24.536807 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eea2119-4be9-493d-b7f0-85e9c3e9381c None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:24.538367 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:24.552089 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-a0c0071a-3dbe-4c80-98e6-1d96ae1d0e9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:24.552089 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:24.552089 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:24.552089 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:24.552089 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:24.552089 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:24.552089 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:24.552089 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:24.552089 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:24.552089 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:24.552089 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:24.552089 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:24.552089 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:29:24.552089 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:29:24.552089 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:29:24.553146 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:29:24.553146 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:24.553146 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:24.553146 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:24.553146 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:24.553146 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:24.553146 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:24.553146 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:24.553146 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:24.553146 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:24.553146 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:24.553146 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:24.553146 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:24.553146 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:24.553146 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:24.553146 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:24.553146 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:24.554062 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:29:24.554062 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:29:24.554062 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:24.554062 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:24.554062 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 435/851] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:29:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:24.554354 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c0071a-3dbe-4c80-98e6-1d96ae1d0e9d None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:24.575430 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c74d72-1508-4daf-9bb6-27cb5b4ae834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:24.576085 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c74d72-1508-4daf-9bb6-27cb5b4ae834 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:24.576372 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c74d72-1508-4daf-9bb6-27cb5b4ae834 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:24.576645 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c74d72-1508-4daf-9bb6-27cb5b4ae834 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:24.630964 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:24.648073 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c74d72-1508-4daf-9bb6-27cb5b4ae834 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:24.649720 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c74d72-1508-4daf-9bb6-27cb5b4ae834 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:24.657928 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c0071a-3dbe-4c80-98e6-1d96ae1d0e9d None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:24.701026 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf26ee-64f0-4383-bff0-d326394c2ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:24.701751 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf26ee-64f0-4383-bff0-d326394c2ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:24.702041 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf26ee-64f0-4383-bff0-d326394c2ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:24.702330 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf26ee-64f0-4383-bff0-d326394c2ee6 None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:24.703056 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bf26ee-64f0-4383-bff0-d326394c2ee6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:24.703364 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bf26ee-64f0-4383-bff0-d326394c2ee6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:24.704767 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bf26ee-64f0-4383-bff0-d326394c2ee6 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:24.709284 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 436/852] 10.222.0.36 () {56 vars in 1114 bytes} [Thu May 19 16:29:24 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:24.710064 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf26ee-64f0-4383-bff0-d326394c2ee6 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:24.764525 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 417/853] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:29:24 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 16:29:24.765355 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c74d72-1508-4daf-9bb6-27cb5b4ae834 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:24.768214 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:24.832155 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf26ee-64f0-4383-bff0-d326394c2ee6 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:24.875709 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04280dd3-edba-4c98-b93f-01510cbc23b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:24.876667 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04280dd3-edba-4c98-b93f-01510cbc23b0 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:24.876958 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04280dd3-edba-4c98-b93f-01510cbc23b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:24.877273 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04280dd3-edba-4c98-b93f-01510cbc23b0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:24.877949 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04280dd3-edba-4c98-b93f-01510cbc23b0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:24.879609 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04280dd3-edba-4c98-b93f-01510cbc23b0 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:24.882208 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c74d72-1508-4daf-9bb6-27cb5b4ae834 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:24.893345 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-04280dd3-edba-4c98-b93f-01510cbc23b0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 16:29:24.893345 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:24.893345 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 16:29:24.893345 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 16:29:24.893345 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 16:29:24.893345 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 16:29:24.893345 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 16:29:24.893345 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:29:24.893345 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 16:29:24.893345 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:24.893345 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 16:29:24.893345 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:24.893345 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:24.893345 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:24.893345 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:24.893345 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:24.894277 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:24.894277 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:24.894277 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:24.894277 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:24.894277 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:24.894277 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:24.894277 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:24.894277 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 16:29:24.894277 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 16:29:24.894277 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 16:29:24.894277 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 16:29:24.894277 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:24.894277 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:24.894277 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:24.894277 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:24.894277 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:24.895395 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:24.895395 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:24.895395 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:24.895395 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:24.895395 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:24.895395 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:24.895395 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:24.895395 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:24.895395 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:24.895395 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:24.895395 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:24.895395 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 16:29:24.895395 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:29:24.895395 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 16:29:24.895395 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:24.895395 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 437/854] 10.222.0.36 () {56 vars in 1132 bytes} [Thu May 19 16:29:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:24.896263 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04280dd3-edba-4c98-b93f-01510cbc23b0 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:24.899952 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:24.926000 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a2a84-d14e-4f0a-a128-159ae5542b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:24.926709 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a2a84-d14e-4f0a-a128-159ae5542b82 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:24.926974 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a2a84-d14e-4f0a-a128-159ae5542b82 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:24.927260 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a2a84-d14e-4f0a-a128-159ae5542b82 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:24.928016 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248a2a84-d14e-4f0a-a128-159ae5542b82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:24.928334 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248a2a84-d14e-4f0a-a128-159ae5542b82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:24.929778 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248a2a84-d14e-4f0a-a128-159ae5542b82 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:24.943526 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 418/855] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:29:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:24.945000 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a2a84-d14e-4f0a-a128-159ae5542b82 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:25.024778 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:25.030039 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04280dd3-edba-4c98-b93f-01510cbc23b0 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:25.067333 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff10429-b348-4326-979f-eb9d1c132df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:25.067572 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a2a84-d14e-4f0a-a128-159ae5542b82 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:25.068128 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff10429-b348-4326-979f-eb9d1c132df8 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:25.068411 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff10429-b348-4326-979f-eb9d1c132df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:25.068685 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff10429-b348-4326-979f-eb9d1c132df8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:25.083643 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff10429-b348-4326-979f-eb9d1c132df8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:25.085409 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff10429-b348-4326-979f-eb9d1c132df8 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:25.104228 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8a1490-eaa3-4dc0-805e-d4204b5f0d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:25.104901 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a1490-eaa3-4dc0-805e-d4204b5f0d28 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:25.105209 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a1490-eaa3-4dc0-805e-d4204b5f0d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:25.105497 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a1490-eaa3-4dc0-805e-d4204b5f0d28 None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:25.105670 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-0ff10429-b348-4326-979f-eb9d1c132df8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:25.105670 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:25.105670 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:25.105670 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:25.105670 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:25.105670 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:25.105670 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:25.105670 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:25.105670 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:25.105670 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:25.105670 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:25.105670 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:25.105670 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:29:25.105670 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:29:25.105670 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:29:25.106500 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:29:25.106500 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:25.106500 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:25.106500 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:25.106500 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:25.106500 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:25.106500 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:25.106500 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:25.106500 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:25.106500 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:25.106500 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:25.106500 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:25.106500 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:25.106500 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:25.106500 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:25.106500 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:25.106500 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:25.107529 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:29:25.107529 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:29:25.107529 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:25.107529 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:25.107747 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8a1490-eaa3-4dc0-805e-d4204b5f0d28 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:25.107747 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8a1490-eaa3-4dc0-805e-d4204b5f0d28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:25.109207 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8a1490-eaa3-4dc0-805e-d4204b5f0d28 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:25.111692 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 438/856] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:29:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:25.112241 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff10429-b348-4326-979f-eb9d1c132df8 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:25.119447 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 419/857] 10.222.0.36 () {56 vars in 1126 bytes} [Thu May 19 16:29:24 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:25.119447 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8a1490-eaa3-4dc0-805e-d4204b5f0d28 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:25.177712 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:25.226258 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8a1490-eaa3-4dc0-805e-d4204b5f0d28 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:25.236324 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff10429-b348-4326-979f-eb9d1c132df8 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:25.257028 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:25.269099 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3789ccb5-c114-46ff-939c-0849944ccb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:25.270133 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3789ccb5-c114-46ff-939c-0849944ccb48 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:25.270430 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3789ccb5-c114-46ff-939c-0849944ccb48 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:25.270704 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3789ccb5-c114-46ff-939c-0849944ccb48 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:25.271435 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3789ccb5-c114-46ff-939c-0849944ccb48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:25.271740 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3789ccb5-c114-46ff-939c-0849944ccb48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:25.273133 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3789ccb5-c114-46ff-939c-0849944ccb48 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:25.273301 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8def56-f47e-4e44-87ac-6d42f0f4531f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:25.274073 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8def56-f47e-4e44-87ac-6d42f0f4531f None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:25.274360 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8def56-f47e-4e44-87ac-6d42f0f4531f None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:25.274643 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8def56-f47e-4e44-87ac-6d42f0f4531f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:25.283700 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8def56-f47e-4e44-87ac-6d42f0f4531f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:25.285087 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8def56-f47e-4e44-87ac-6d42f0f4531f None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:25.291993 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 420/858] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:29:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:25.291993 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3789ccb5-c114-46ff-939c-0849944ccb48 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:25.306347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-9f8def56-f47e-4e44-87ac-6d42f0f4531f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 16:29:25.306347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:25.306347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:25.306347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:25.306347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:25.306347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:25.306347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:25.306347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:25.306347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:25.306347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:25.306347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:25.306347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:25.306347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 16:29:25.306347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 16:29:25.306347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 16:29:25.306347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 16:29:25.307507 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:25.307507 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:25.307507 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 16:29:25.307507 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:29:25.307507 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 16:29:25.307507 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:29:25.307507 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:25.307507 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:25.307507 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:25.307507 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:25.307507 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:25.307507 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:25.307507 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:25.307507 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:25.307507 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:25.307507 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:25.307507 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:25.307507 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:25.308488 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:25.308488 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:25.308488 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 16:29:25.308488 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 16:29:25.308488 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 16:29:25.308488 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 16:29:25.308488 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 16:29:25.308488 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 16:29:25.308488 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 16:29:25.308488 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:25.308488 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 439/859] 10.222.0.36 () {56 vars in 1117 bytes} [Thu May 19 16:29:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:25.308488 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8def56-f47e-4e44-87ac-6d42f0f4531f None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:25.415639 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3789ccb5-c114-46ff-939c-0849944ccb48 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:25.425566 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8def56-f47e-4e44-87ac-6d42f0f4531f None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:25.440199 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:25.457628 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510f076d-e522-465f-9edf-d8d9767fe802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:25.458704 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f076d-e522-465f-9edf-d8d9767fe802 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:25.459255 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f076d-e522-465f-9edf-d8d9767fe802 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:25.459537 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f076d-e522-465f-9edf-d8d9767fe802 None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:25.460265 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510f076d-e522-465f-9edf-d8d9767fe802 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:25.460576 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510f076d-e522-465f-9edf-d8d9767fe802 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:25.461975 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510f076d-e522-465f-9edf-d8d9767fe802 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:25.462930 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df87c82-bfb3-4bf5-9de5-8689b95ef0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:25.463742 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df87c82-bfb3-4bf5-9de5-8689b95ef0b2 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:25.464034 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df87c82-bfb3-4bf5-9de5-8689b95ef0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:25.464319 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df87c82-bfb3-4bf5-9de5-8689b95ef0b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:25.468188 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 421/860] 10.222.0.36 () {56 vars in 1126 bytes} [Thu May 19 16:29:25 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:25.469445 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510f076d-e522-465f-9edf-d8d9767fe802 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:25.471850 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df87c82-bfb3-4bf5-9de5-8689b95ef0b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:25.473403 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df87c82-bfb3-4bf5-9de5-8689b95ef0b2 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:25.496823 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-6df87c82-bfb3-4bf5-9de5-8689b95ef0b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:29:25.496823 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:25.496823 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:25.496823 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:25.496823 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:25.496823 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:25.496823 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:25.496823 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:25.496823 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:25.496823 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:25.496823 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:25.496823 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:25.496823 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 16:29:25.496823 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 16:29:25.496823 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 16:29:25.498097 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 16:29:25.498097 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:25.498097 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:25.498097 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:25.498097 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:25.498097 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:25.498097 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:25.498097 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:25.498097 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:25.498097 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:25.498097 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:25.498097 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:25.498097 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:25.498097 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:25.498097 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:25.498097 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:25.498097 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:25.498994 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 16:29:25.498994 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 16:29:25.498994 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 16:29:25.498994 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 16:29:25.498994 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 16:29:25.498994 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 16:29:25.498994 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:29:25.498994 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:25.504029 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 440/861] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:29:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:25.505067 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df87c82-bfb3-4bf5-9de5-8689b95ef0b2 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:25.545000 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:25.618771 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510f076d-e522-465f-9edf-d8d9767fe802 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:25.654036 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df87c82-bfb3-4bf5-9de5-8689b95ef0b2 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:25.673880 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a3211-fc5c-4752-b7a9-4859840ba08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:25.674569 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a3211-fc5c-4752-b7a9-4859840ba08d None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:25.674854 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a3211-fc5c-4752-b7a9-4859840ba08d None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:25.675139 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a3211-fc5c-4752-b7a9-4859840ba08d None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:25.676002 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a3211-fc5c-4752-b7a9-4859840ba08d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:25.676352 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a3211-fc5c-4752-b7a9-4859840ba08d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:25.678242 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a3211-fc5c-4752-b7a9-4859840ba08d None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:25.699843 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 422/862] 10.222.0.36 () {56 vars in 1149 bytes} [Thu May 19 16:29:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:25.703831 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c573ec-ba8a-49ad-8b45-db7050430edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:25.704512 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c573ec-ba8a-49ad-8b45-db7050430edf None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:25.704797 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c573ec-ba8a-49ad-8b45-db7050430edf None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:25.705312 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c573ec-ba8a-49ad-8b45-db7050430edf None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:25.709202 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a3211-fc5c-4752-b7a9-4859840ba08d None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:25.709360 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c573ec-ba8a-49ad-8b45-db7050430edf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:25.709360 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c573ec-ba8a-49ad-8b45-db7050430edf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:25.709360 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c573ec-ba8a-49ad-8b45-db7050430edf None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:25.723157 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 441/863] 10.222.0.36 () {56 vars in 1148 bytes} [Thu May 19 16:29:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:25.728697 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c573ec-ba8a-49ad-8b45-db7050430edf None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:25.770918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:25.823746 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a3211-fc5c-4752-b7a9-4859840ba08d None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:25.838377 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:25.868226 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bac648-6f07-4d4c-9174-b038c3a08a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:25.869112 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bac648-6f07-4d4c-9174-b038c3a08a9b None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:25.869491 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bac648-6f07-4d4c-9174-b038c3a08a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:25.869838 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bac648-6f07-4d4c-9174-b038c3a08a9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:25.870743 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c573ec-ba8a-49ad-8b45-db7050430edf None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:25.925228 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530a3812-8560-49ad-a18d-2560e7ad6ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:25.925913 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a3812-8560-49ad-a18d-2560e7ad6ec1 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:25.926200 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a3812-8560-49ad-a18d-2560e7ad6ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:25.926483 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a3812-8560-49ad-a18d-2560e7ad6ec1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:25.927335 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530a3812-8560-49ad-a18d-2560e7ad6ec1 None admin] RBAC: Adding query filter params (role.id=e2915fe109244a07a9be96bd2b219fdd, scope.project.id=631c76794a124d6da22c626e590e4352, user.id=ffe7cebc6f3b40d8a8e1866ed2da9822) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:25.927652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530a3812-8560-49ad-a18d-2560e7ad6ec1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:25.929111 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530a3812-8560-49ad-a18d-2560e7ad6ec1 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:25.935826 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 442/864] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:29:25 2022] GET /identity/v3/role_assignments?user.id=ffe7cebc6f3b40d8a8e1866ed2da9822&scope.project.id=631c76794a124d6da22c626e590e4352&role.id=e2915fe109244a07a9be96bd2b219fdd => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:25.970314 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bac648-6f07-4d4c-9174-b038c3a08a9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:25.972067 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bac648-6f07-4d4c-9174-b038c3a08a9b None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:26.093975 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 423/865] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:29:25 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 16:29:26.100327 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530a3812-8560-49ad-a18d-2560e7ad6ec1 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:26.212414 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:26.217629 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530a3812-8560-49ad-a18d-2560e7ad6ec1 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:26.253683 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03512e08-f1b8-4cbb-9ec5-fe0bfffa3f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:26.254485 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03512e08-f1b8-4cbb-9ec5-fe0bfffa3f1d None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:26.254770 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03512e08-f1b8-4cbb-9ec5-fe0bfffa3f1d None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:26.255047 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03512e08-f1b8-4cbb-9ec5-fe0bfffa3f1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:26.266107 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03512e08-f1b8-4cbb-9ec5-fe0bfffa3f1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:26.267568 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03512e08-f1b8-4cbb-9ec5-fe0bfffa3f1d None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:26.286578 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-03512e08-f1b8-4cbb-9ec5-fe0bfffa3f1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:26.286578 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:26.286578 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:26.286578 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:26.286578 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:26.286578 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:26.286578 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:26.286578 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:26.286578 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:26.286578 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:26.286578 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:26.286578 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:26.286578 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:29:26.286578 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:29:26.286578 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:29:26.287795 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:29:26.287795 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:26.287795 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:26.287795 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:26.287795 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:26.287795 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:26.287795 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:26.287795 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:26.287795 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:26.287795 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:26.287795 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:26.287795 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:26.287795 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:26.287795 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:26.287795 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:26.287795 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:26.287795 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:26.288819 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:29:26.288819 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:29:26.288819 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:26.288819 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:26.292860 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 443/866] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:29:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:26.294384 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bac648-6f07-4d4c-9174-b038c3a08a9b None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:26.309501 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:26.399379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bac648-6f07-4d4c-9174-b038c3a08a9b None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:26.435482 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd8511d-53ca-4c10-b7f2-25499b1eda46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:26.436160 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd8511d-53ca-4c10-b7f2-25499b1eda46 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:26.436448 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd8511d-53ca-4c10-b7f2-25499b1eda46 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:26.436727 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd8511d-53ca-4c10-b7f2-25499b1eda46 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:26.437551 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd8511d-53ca-4c10-b7f2-25499b1eda46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:26.437860 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd8511d-53ca-4c10-b7f2-25499b1eda46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:26.439239 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd8511d-53ca-4c10-b7f2-25499b1eda46 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:26.451790 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 424/867] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:29:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:29:26.457015 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03512e08-f1b8-4cbb-9ec5-fe0bfffa3f1d None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:26.563597 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03512e08-f1b8-4cbb-9ec5-fe0bfffa3f1d None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:26.600095 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd24ae-2941-44a1-9c0f-a34e9bc3aa99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:26.601302 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd24ae-2941-44a1-9c0f-a34e9bc3aa99 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:26.601811 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd24ae-2941-44a1-9c0f-a34e9bc3aa99 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:26.602295 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd24ae-2941-44a1-9c0f-a34e9bc3aa99 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:26.613681 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bd24ae-2941-44a1-9c0f-a34e9bc3aa99 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:26.615515 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bd24ae-2941-44a1-9c0f-a34e9bc3aa99 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:26.631075 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-18bd24ae-2941-44a1-9c0f-a34e9bc3aa99 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:29:26.631075 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:26.631075 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:26.631075 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:26.631075 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:26.631075 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:26.631075 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:26.631075 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:26.631075 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:26.631075 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:26.631075 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:26.631075 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:26.631075 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 16:29:26.631075 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 16:29:26.631075 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 16:29:26.632048 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 16:29:26.632048 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:26.632048 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:26.632048 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:26.632048 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:26.632048 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:26.632048 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:26.632048 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:26.632048 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:26.632048 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:26.632048 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:26.632048 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:26.632048 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:26.632048 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:26.632048 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:26.632048 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:26.632048 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:26.632947 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 16:29:26.632947 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 16:29:26.632947 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 16:29:26.632947 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 16:29:26.632947 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 16:29:26.632947 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 16:29:26.632947 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:29:26.632947 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:26.635508 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 444/868] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:29:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:26.650753 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd8511d-53ca-4c10-b7f2-25499b1eda46 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:26.725992 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:26.768596 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd8511d-53ca-4c10-b7f2-25499b1eda46 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:26.805359 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3856ffa1-ab8b-45f7-8546-046f1f24d59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:26.806033 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3856ffa1-ab8b-45f7-8546-046f1f24d59b None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:26.806321 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3856ffa1-ab8b-45f7-8546-046f1f24d59b None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:26.806600 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3856ffa1-ab8b-45f7-8546-046f1f24d59b None admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:26.807404 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3856ffa1-ab8b-45f7-8546-046f1f24d59b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:26.807714 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3856ffa1-ab8b-45f7-8546-046f1f24d59b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:26.809180 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3856ffa1-ab8b-45f7-8546-046f1f24d59b None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:26.822407 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 425/869] 10.222.0.36 () {56 vars in 1148 bytes} [Thu May 19 16:29:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:26.828671 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd24ae-2941-44a1-9c0f-a34e9bc3aa99 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:26.856001 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:26.938323 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd24ae-2941-44a1-9c0f-a34e9bc3aa99 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:26.987369 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb04ee8-829c-44da-9852-47e1b1a317f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:26.988413 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb04ee8-829c-44da-9852-47e1b1a317f5 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:26.988941 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb04ee8-829c-44da-9852-47e1b1a317f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:26.989461 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb04ee8-829c-44da-9852-47e1b1a317f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:26.990566 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb04ee8-829c-44da-9852-47e1b1a317f5 None admin] RBAC: Adding query filter params (role.id=727026b648e1429ca3697e04800b328a, scope.project.id=631c76794a124d6da22c626e590e4352, user.id=a30dc1dd2b00463bb3c7957864c56043) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:26.991091 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb04ee8-829c-44da-9852-47e1b1a317f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:26.992777 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb04ee8-829c-44da-9852-47e1b1a317f5 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:27.000739 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 445/870] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:29:26 2022] GET /identity/v3/role_assignments?user.id=a30dc1dd2b00463bb3c7957864c56043&scope.project.id=631c76794a124d6da22c626e590e4352&role.id=727026b648e1429ca3697e04800b328a => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:27.809960 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378a02f-a71c-4d91-9edf-ec9c665dfe4a None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:27.810494 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378a02f-a71c-4d91-9edf-ec9c665dfe4a None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:27.810980 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378a02f-a71c-4d91-9edf-ec9c665dfe4a None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:27.814243 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 426/871] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:29:27.822978 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd220bd-2542-4388-bcce-bd299bdf75e1 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:27.823824 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd220bd-2542-4388-bcce-bd299bdf75e1 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:27.823824 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd220bd-2542-4388-bcce-bd299bdf75e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:27.853228 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-ebd220bd-2542-4388-bcce-bd299bdf75e1 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:27.855789 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-ebd220bd-2542-4388-bcce-bd299bdf75e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:27.895618 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e55da-316d-479a-8bc8-1f2327302ac0 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:27.900261 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e55da-316d-479a-8bc8-1f2327302ac0 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:27.901031 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e55da-316d-479a-8bc8-1f2327302ac0 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:27.904398 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 427/872] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:27 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:29:27.911559 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643303c9-baca-4ba0-86fc-553367cb5c21 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:27.911926 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643303c9-baca-4ba0-86fc-553367cb5c21 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:27.912232 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643303c9-baca-4ba0-86fc-553367cb5c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:27.929877 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-643303c9-baca-4ba0-86fc-553367cb5c21 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:27.931072 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-643303c9-baca-4ba0-86fc-553367cb5c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:27.979091 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 446/873] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:27 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:29:28.007450 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd220bd-2542-4388-bcce-bd299bdf75e1 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:28.010812 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 428/874] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:27 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 16:29:28.011769 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:28.034059 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643303c9-baca-4ba0-86fc-553367cb5c21 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:28.097646 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:28.142377 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd220bd-2542-4388-bcce-bd299bdf75e1 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:28.153890 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643303c9-baca-4ba0-86fc-553367cb5c21 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:28.180432 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7da675-8385-4388-9ba8-0772d64adf3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:28.181126 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7da675-8385-4388-9ba8-0772d64adf3f None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:28.181458 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7da675-8385-4388-9ba8-0772d64adf3f None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:28.181741 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7da675-8385-4388-9ba8-0772d64adf3f None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:28.182513 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7da675-8385-4388-9ba8-0772d64adf3f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:28.184365 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7da675-8385-4388-9ba8-0772d64adf3f None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:28.209742 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 447/875] 10.222.0.36 () {60 vars in 1141 bytes} [Thu May 19 16:29:28 2022] POST /identity/v3/services => generated 251 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:29:28.218460 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35963aa-cd7b-416a-9300-4990badf1118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:28.219575 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35963aa-cd7b-416a-9300-4990badf1118 None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:28.220074 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35963aa-cd7b-416a-9300-4990badf1118 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:28.220599 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35963aa-cd7b-416a-9300-4990badf1118 None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:28.221848 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35963aa-cd7b-416a-9300-4990badf1118 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:28.223568 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35963aa-cd7b-416a-9300-4990badf1118 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:28.244141 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 429/876] 10.222.0.36 () {60 vars in 1141 bytes} [Thu May 19 16:29:28 2022] POST /identity/v3/services => generated 269 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:29:28.400086 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4374e4-4e69-4ec5-bb6a-007e4257d519 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:28.400475 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4374e4-4e69-4ec5-bb6a-007e4257d519 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:28.400677 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4374e4-4e69-4ec5-bb6a-007e4257d519 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:28.402591 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 448/877] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:29:28.409316 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e248a-e829-443f-92fd-ab0f2bb20cf4 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:28.409640 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e248a-e829-443f-92fd-ab0f2bb20cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:28.409899 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e248a-e829-443f-92fd-ab0f2bb20cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:28.429110 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-fc7e248a-e829-443f-92fd-ab0f2bb20cf4 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:28.430416 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-fc7e248a-e829-443f-92fd-ab0f2bb20cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:28.518462 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 430/878] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:28 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:29:28.541661 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4374e4-4e69-4ec5-bb6a-007e4257d519 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:28.620288 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:28.665848 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4374e4-4e69-4ec5-bb6a-007e4257d519 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:28.702293 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319bc347-0c7b-4ee5-bd8f-160d099e3fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:28.703162 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319bc347-0c7b-4ee5-bd8f-160d099e3fef None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:28.703449 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319bc347-0c7b-4ee5-bd8f-160d099e3fef None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:28.703730 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319bc347-0c7b-4ee5-bd8f-160d099e3fef None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:28.704410 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319bc347-0c7b-4ee5-bd8f-160d099e3fef None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:28.705853 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319bc347-0c7b-4ee5-bd8f-160d099e3fef None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:28.719273 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-319bc347-0c7b-4ee5-bd8f-160d099e3fef None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 16:29:28.719273 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:28.719273 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 16:29:28.719273 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 16:29:28.719273 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 16:29:28.719273 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 16:29:28.719273 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 16:29:28.719273 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:29:28.719273 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 16:29:28.719273 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:28.719273 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 16:29:28.719273 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:28.719273 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:28.719273 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:28.719273 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:28.719273 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:28.719273 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:28.720305 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:28.720305 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:28.720305 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:28.720305 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:28.720305 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:28.720305 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:28.720305 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 16:29:28.720305 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 16:29:28.720305 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 16:29:28.720305 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 16:29:28.720305 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:28.720305 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:28.720305 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:28.720305 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:28.720305 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:28.720305 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:28.721137 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:28.721137 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:28.721137 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:28.721137 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:28.721137 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:28.721137 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:28.721137 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:28.721137 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:28.721137 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:28.721137 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:28.721137 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 16:29:28.721137 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:29:28.721137 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 16:29:28.721137 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:28.721137 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 449/879] 10.222.0.36 () {56 vars in 1126 bytes} [Thu May 19 16:29:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:28.726834 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e248a-e829-443f-92fd-ab0f2bb20cf4 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:28.772909 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:28.837414 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e248a-e829-443f-92fd-ab0f2bb20cf4 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:28.874243 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c2829-deb1-4cf8-94d7-157e6e9f0179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:28.874956 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c2829-deb1-4cf8-94d7-157e6e9f0179 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:28.875237 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c2829-deb1-4cf8-94d7-157e6e9f0179 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:28.875512 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c2829-deb1-4cf8-94d7-157e6e9f0179 None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:28.876263 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58c2829-deb1-4cf8-94d7-157e6e9f0179 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:28.876568 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58c2829-deb1-4cf8-94d7-157e6e9f0179 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:28.877985 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58c2829-deb1-4cf8-94d7-157e6e9f0179 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:28.887743 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 431/880] 10.222.0.36 () {56 vars in 1122 bytes} [Thu May 19 16:29:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:28.889113 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319bc347-0c7b-4ee5-bd8f-160d099e3fef None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:28.997678 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319bc347-0c7b-4ee5-bd8f-160d099e3fef None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:29.032043 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:29.041079 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3943481d-88e7-4de5-884f-6bb455761026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:29.041795 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3943481d-88e7-4de5-884f-6bb455761026 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:29.042483 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3943481d-88e7-4de5-884f-6bb455761026 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:29.043164 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3943481d-88e7-4de5-884f-6bb455761026 None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:29.043890 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3943481d-88e7-4de5-884f-6bb455761026 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:29.044200 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3943481d-88e7-4de5-884f-6bb455761026 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:29.045621 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3943481d-88e7-4de5-884f-6bb455761026 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:29.056890 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 450/881] 10.222.0.36 () {56 vars in 1122 bytes} [Thu May 19 16:29:28 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:29.361115 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2bc409-51cc-404f-a5d3-e9bee7ebf2a2 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:29.361502 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2bc409-51cc-404f-a5d3-e9bee7ebf2a2 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:29.361682 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2bc409-51cc-404f-a5d3-e9bee7ebf2a2 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:29.363316 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 432/882] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:29:29.368706 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18937e4d-289a-4f2a-91f4-22c83fa70aa4 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:29.368995 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18937e4d-289a-4f2a-91f4-22c83fa70aa4 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:29.369301 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18937e4d-289a-4f2a-91f4-22c83fa70aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:29.387264 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-18937e4d-289a-4f2a-91f4-22c83fa70aa4 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:29.388570 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-18937e4d-289a-4f2a-91f4-22c83fa70aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:29.470579 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 451/883] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:29 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:29:29.494502 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2bc409-51cc-404f-a5d3-e9bee7ebf2a2 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:29.624277 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2bc409-51cc-404f-a5d3-e9bee7ebf2a2 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:29.627193 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:29.663109 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb942552-27f2-4300-9b07-4a9fd8bee0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:29.664420 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb942552-27f2-4300-9b07-4a9fd8bee0ed None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:29.664940 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb942552-27f2-4300-9b07-4a9fd8bee0ed None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:29.665462 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb942552-27f2-4300-9b07-4a9fd8bee0ed None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:29.676983 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb942552-27f2-4300-9b07-4a9fd8bee0ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:29:29.685627 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-eb942552-27f2-4300-9b07-4a9fd8bee0ed None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 16:29:29.685627 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:29.685627 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:29.685627 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:29.685627 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:29.685627 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:29.685627 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:29.685627 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:29.685627 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:29.685627 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:29.685627 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:29.685627 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:29.685627 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:29:29.685627 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:29:29.685627 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:29:29.685627 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:29:29.686582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:29:29.686582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:29:29.686582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:29:29.686582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:29:29.686582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:29.686582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:29.686582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:29.686582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:29.686582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:29.686582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:29.686582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:29.686582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:29.686582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:29.686582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:29.686582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:29.686582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:29.687463 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:29.687463 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:29.687463 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:29.687463 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:29.687463 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:29:29.687463 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:29:29.687463 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:29:29.687463 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:29:29.687463 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:29:29.687463 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:29:29.687463 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 16:29:29.687463 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:29.690213 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 433/884] 10.222.0.36 () {56 vars in 1120 bytes} [Thu May 19 16:29:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:29.696229 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18937e4d-289a-4f2a-91f4-22c83fa70aa4 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:29.801992 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18937e4d-289a-4f2a-91f4-22c83fa70aa4 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:29.825519 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:29.838624 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264da81c-390a-4367-a2df-3cb5c0d84500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:29.839307 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264da81c-390a-4367-a2df-3cb5c0d84500 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:29.839592 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264da81c-390a-4367-a2df-3cb5c0d84500 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:29.839871 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264da81c-390a-4367-a2df-3cb5c0d84500 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:29.840628 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264da81c-390a-4367-a2df-3cb5c0d84500 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:29.840941 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264da81c-390a-4367-a2df-3cb5c0d84500 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:29.842361 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264da81c-390a-4367-a2df-3cb5c0d84500 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:29.852307 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 452/885] 10.222.0.36 () {56 vars in 1115 bytes} [Thu May 19 16:29:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:29.857964 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb942552-27f2-4300-9b07-4a9fd8bee0ed None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:29.967614 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb942552-27f2-4300-9b07-4a9fd8bee0ed None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:30.003734 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb07b5b-76e5-49a0-81b0-d9dd312e148c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:30.004888 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb07b5b-76e5-49a0-81b0-d9dd312e148c None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:30.005438 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb07b5b-76e5-49a0-81b0-d9dd312e148c None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:30.005924 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb07b5b-76e5-49a0-81b0-d9dd312e148c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:30.017715 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb07b5b-76e5-49a0-81b0-d9dd312e148c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:30.019722 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb07b5b-76e5-49a0-81b0-d9dd312e148c None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:30.037314 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-ceb07b5b-76e5-49a0-81b0-d9dd312e148c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:30.037314 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:30.037314 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:30.037314 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:30.037314 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:30.037314 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:30.037314 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:30.037314 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:30.037314 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:30.037314 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:30.037314 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:30.037314 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:30.037314 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:29:30.037314 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:29:30.037314 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:29:30.038259 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:29:30.038259 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:30.038259 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:30.038259 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:30.038259 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:30.038259 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:30.038259 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:30.038259 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:30.038259 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:30.038259 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:30.038259 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:30.038259 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:30.038259 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:30.038259 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:30.038259 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:30.038259 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:30.038259 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:30.039233 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:29:30.039233 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:29:30.039233 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:30.039233 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:30.041549 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 434/886] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:29:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:30.047463 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264da81c-390a-4367-a2df-3cb5c0d84500 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:30.167323 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264da81c-390a-4367-a2df-3cb5c0d84500 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:30.206766 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42fa5ad-25a2-4872-a76c-2a5375a87d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:30.207457 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42fa5ad-25a2-4872-a76c-2a5375a87d6c None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:30.207741 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42fa5ad-25a2-4872-a76c-2a5375a87d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:30.208021 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42fa5ad-25a2-4872-a76c-2a5375a87d6c None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:30.208756 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42fa5ad-25a2-4872-a76c-2a5375a87d6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:30.209062 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42fa5ad-25a2-4872-a76c-2a5375a87d6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:30.210530 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42fa5ad-25a2-4872-a76c-2a5375a87d6c None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:30.224564 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 453/887] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:29:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:30.230148 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb07b5b-76e5-49a0-81b0-d9dd312e148c None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:30.251064 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:30.340027 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb07b5b-76e5-49a0-81b0-d9dd312e148c None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:30.376629 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d062fcaf-bf44-41bb-be3a-960c553dff7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:30.377872 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062fcaf-bf44-41bb-be3a-960c553dff7e None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:30.378387 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062fcaf-bf44-41bb-be3a-960c553dff7e None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:30.378880 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062fcaf-bf44-41bb-be3a-960c553dff7e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:30.388138 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d062fcaf-bf44-41bb-be3a-960c553dff7e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:30.390485 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d062fcaf-bf44-41bb-be3a-960c553dff7e None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:30.406561 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-d062fcaf-bf44-41bb-be3a-960c553dff7e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 16:29:30.406561 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:30.406561 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:30.406561 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:30.406561 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:30.406561 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:30.406561 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:30.406561 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:30.406561 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:30.406561 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:30.406561 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:30.406561 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:30.406561 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 16:29:30.406561 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 16:29:30.406561 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 16:29:30.406561 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 16:29:30.407540 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:30.407540 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:30.407540 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 16:29:30.407540 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:29:30.407540 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 16:29:30.407540 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:29:30.407540 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:30.407540 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:30.407540 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:30.407540 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:30.407540 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:30.407540 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:30.407540 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:30.407540 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:30.407540 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:30.407540 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:30.407540 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:30.407540 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:30.408488 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:30.408488 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:30.408488 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 16:29:30.408488 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 16:29:30.408488 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 16:29:30.408488 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 16:29:30.408488 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 16:29:30.408488 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 16:29:30.408488 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 16:29:30.408488 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:30.411143 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 435/888] 10.222.0.36 () {56 vars in 1117 bytes} [Thu May 19 16:29:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:30.416945 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42fa5ad-25a2-4872-a76c-2a5375a87d6c None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:30.476171 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:30.540337 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42fa5ad-25a2-4872-a76c-2a5375a87d6c None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:30.575140 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b577f15-4091-4c90-bca5-4053a2a5df0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:30.575822 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b577f15-4091-4c90-bca5-4053a2a5df0d None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:30.576105 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b577f15-4091-4c90-bca5-4053a2a5df0d None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:30.576381 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b577f15-4091-4c90-bca5-4053a2a5df0d None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:30.577252 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b577f15-4091-4c90-bca5-4053a2a5df0d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:30.578127 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b577f15-4091-4c90-bca5-4053a2a5df0d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:30.579563 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b577f15-4091-4c90-bca5-4053a2a5df0d None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:30.598486 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 454/889] 10.222.0.36 () {56 vars in 1149 bytes} [Thu May 19 16:29:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:30.604150 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d062fcaf-bf44-41bb-be3a-960c553dff7e None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:30.646027 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d70a9c-a4c6-42a3-b2b9-2a38d7536243 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:30.646672 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d70a9c-a4c6-42a3-b2b9-2a38d7536243 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:30.647177 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d70a9c-a4c6-42a3-b2b9-2a38d7536243 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:30.649526 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 455/890] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 16:29:30.652241 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933b1fb-e0cc-4be1-83b3-6d7734ae39f7 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:30.653058 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933b1fb-e0cc-4be1-83b3-6d7734ae39f7 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:30.653888 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933b1fb-e0cc-4be1-83b3-6d7734ae39f7 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:30.655464 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 456/891] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:29:30.656791 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b6f96-500c-4179-9559-20b47190f95a None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:30.657381 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b6f96-500c-4179-9559-20b47190f95a None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:30.657906 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b6f96-500c-4179-9559-20b47190f95a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:30.675183 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-c33b6f96-500c-4179-9559-20b47190f95a None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:30.676393 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-c33b6f96-500c-4179-9559-20b47190f95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:30.689224 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:30.714438 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d062fcaf-bf44-41bb-be3a-960c553dff7e None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:30.749879 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 457/892] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:30 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:29:30.750975 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b36f1-7803-498e-ba9f-8e5b118c151c None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:30.751271 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b36f1-7803-498e-ba9f-8e5b118c151c None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:30.751558 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b36f1-7803-498e-ba9f-8e5b118c151c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:30.754781 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3722a6f-edc8-452d-ae79-4e2cdc699f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:30.755669 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3722a6f-edc8-452d-ae79-4e2cdc699f7a None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:30.756135 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3722a6f-edc8-452d-ae79-4e2cdc699f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:30.756594 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3722a6f-edc8-452d-ae79-4e2cdc699f7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:30.770144 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-cc2b36f1-7803-498e-ba9f-8e5b118c151c None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:30.771517 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-cc2b36f1-7803-498e-ba9f-8e5b118c151c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:30.799625 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:30.829779 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3722a6f-edc8-452d-ae79-4e2cdc699f7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:30.831273 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3722a6f-edc8-452d-ae79-4e2cdc699f7a None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:30.849351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 458/893] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:30 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:29:30.850459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2b36f1-7803-498e-ba9f-8e5b118c151c None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:30.938396 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:30.965920 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 436/894] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:29:30 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 16:29:30.967123 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3722a6f-edc8-452d-ae79-4e2cdc699f7a None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:30.973583 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2b36f1-7803-498e-ba9f-8e5b118c151c None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:31.013460 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd67f8d-bd1a-4095-a2e5-6ca4e5a28d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:31.014545 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd67f8d-bd1a-4095-a2e5-6ca4e5a28d9c None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:31.015020 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd67f8d-bd1a-4095-a2e5-6ca4e5a28d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:31.015438 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd67f8d-bd1a-4095-a2e5-6ca4e5a28d9c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:31.016269 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd67f8d-bd1a-4095-a2e5-6ca4e5a28d9c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:31.017688 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd67f8d-bd1a-4095-a2e5-6ca4e5a28d9c None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:31.030743 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-ddd67f8d-bd1a-4095-a2e5-6ca4e5a28d9c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 16:29:31.030743 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:31.030743 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 16:29:31.030743 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 16:29:31.030743 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 16:29:31.030743 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 16:29:31.030743 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 16:29:31.030743 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:29:31.030743 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 16:29:31.030743 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:31.030743 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 16:29:31.030743 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:31.030743 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:31.030743 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:31.030743 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:31.030743 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:31.031879 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:31.031879 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:31.031879 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:31.031879 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:31.031879 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:31.031879 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:31.031879 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:31.031879 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 16:29:31.031879 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 16:29:31.031879 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 16:29:31.031879 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 16:29:31.031879 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:31.031879 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:31.031879 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:31.031879 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:31.031879 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:31.032868 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:31.032868 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:31.032868 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:31.032868 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:31.032868 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:31.032868 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:31.032868 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:31.032868 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:31.032868 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:31.032868 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:31.032868 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:31.032868 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 16:29:31.032868 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:29:31.032868 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 16:29:31.032868 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:31.034468 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 459/895] 10.222.0.36 () {56 vars in 1132 bytes} [Thu May 19 16:29:30 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:31.035286 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd67f8d-bd1a-4095-a2e5-6ca4e5a28d9c None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:31.071745 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:31.077990 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3722a6f-edc8-452d-ae79-4e2cdc699f7a None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:31.117021 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f72d641-dd73-490e-90d9-18acf6de76c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:31.117944 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72d641-dd73-490e-90d9-18acf6de76c9 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:31.118235 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72d641-dd73-490e-90d9-18acf6de76c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:31.118944 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72d641-dd73-490e-90d9-18acf6de76c9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:31.119617 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f72d641-dd73-490e-90d9-18acf6de76c9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:31.120996 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f72d641-dd73-490e-90d9-18acf6de76c9 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:31.133524 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-9f72d641-dd73-490e-90d9-18acf6de76c9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 16:29:31.133524 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:31.133524 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 16:29:31.133524 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 16:29:31.133524 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 16:29:31.133524 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 16:29:31.133524 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 16:29:31.133524 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:29:31.133524 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 16:29:31.133524 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:31.133524 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 16:29:31.133524 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:31.133524 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:31.133524 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:31.133524 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:31.133524 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:31.133524 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:31.134677 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:31.134677 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:31.134677 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:31.134677 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:31.134677 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:31.134677 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:31.134677 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 16:29:31.134677 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 16:29:31.134677 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 16:29:31.134677 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 16:29:31.134677 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:31.134677 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:31.134677 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:31.134677 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:31.134677 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:31.134677 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:31.135522 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:31.135522 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:31.135522 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:31.135522 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:31.135522 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:31.135522 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:31.135522 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:31.135522 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:31.135522 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:31.135522 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:31.135522 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 16:29:31.135522 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:29:31.135522 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 16:29:31.135522 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:31.137492 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 437/896] 10.222.0.36 () {56 vars in 1114 bytes} [Thu May 19 16:29:30 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:31.138544 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f72d641-dd73-490e-90d9-18acf6de76c9 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:31.157944 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd67f8d-bd1a-4095-a2e5-6ca4e5a28d9c None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:31.184247 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:31.194272 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493332c0-282f-442f-8648-8beeff4ad092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:31.195074 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493332c0-282f-442f-8648-8beeff4ad092 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:31.195373 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493332c0-282f-442f-8648-8beeff4ad092 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:31.195704 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493332c0-282f-442f-8648-8beeff4ad092 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:31.207305 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493332c0-282f-442f-8648-8beeff4ad092 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:31.208776 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493332c0-282f-442f-8648-8beeff4ad092 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:31.226381 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-493332c0-282f-442f-8648-8beeff4ad092 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:31.226381 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:31.226381 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:31.226381 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:31.226381 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:31.226381 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:31.226381 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:31.226381 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:31.226381 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:31.226381 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:31.226381 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:31.226381 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:31.226381 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:29:31.226381 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:29:31.226381 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:29:31.227341 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:29:31.227341 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:31.227341 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:31.227341 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:31.227341 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:31.227341 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:31.227341 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:31.227341 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:31.227341 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:31.227341 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:31.227341 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:31.227341 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:31.227341 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:31.227341 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:31.227341 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:31.227341 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:31.227341 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:31.228244 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:29:31.228244 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:29:31.228244 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:31.228244 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:31.228244 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 460/897] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:29:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:31.228533 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493332c0-282f-442f-8648-8beeff4ad092 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:31.251337 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f72d641-dd73-490e-90d9-18acf6de76c9 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:31.292238 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1659896f-7768-43b7-a987-5fbfd6a55ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:31.293236 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659896f-7768-43b7-a987-5fbfd6a55ef2 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:31.293537 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659896f-7768-43b7-a987-5fbfd6a55ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:31.293882 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659896f-7768-43b7-a987-5fbfd6a55ef2 None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:31.294721 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1659896f-7768-43b7-a987-5fbfd6a55ef2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:31.294973 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1659896f-7768-43b7-a987-5fbfd6a55ef2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:31.296909 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1659896f-7768-43b7-a987-5fbfd6a55ef2 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:31.302714 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 438/898] 10.222.0.36 () {56 vars in 1126 bytes} [Thu May 19 16:29:31 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:31.303547 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1659896f-7768-43b7-a987-5fbfd6a55ef2 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:31.328164 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:31.355700 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493332c0-282f-442f-8648-8beeff4ad092 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:31.400507 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f8404-e19c-45b5-8c4c-969ba079ef67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:31.402051 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f8404-e19c-45b5-8c4c-969ba079ef67 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:31.402631 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f8404-e19c-45b5-8c4c-969ba079ef67 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:31.403792 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f8404-e19c-45b5-8c4c-969ba079ef67 None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:31.405340 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f8404-e19c-45b5-8c4c-969ba079ef67 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:31.406221 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f8404-e19c-45b5-8c4c-969ba079ef67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:31.407810 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f8404-e19c-45b5-8c4c-969ba079ef67 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:31.417757 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 461/899] 10.222.0.36 () {56 vars in 1114 bytes} [Thu May 19 16:29:31 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:31.422284 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f8404-e19c-45b5-8c4c-969ba079ef67 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:31.428393 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1659896f-7768-43b7-a987-5fbfd6a55ef2 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:31.433198 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:31.475709 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd88f6-0b2b-4617-83d6-64710834e557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:31.476363 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd88f6-0b2b-4617-83d6-64710834e557 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:31.476655 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd88f6-0b2b-4617-83d6-64710834e557 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:31.476983 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd88f6-0b2b-4617-83d6-64710834e557 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:31.477903 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cd88f6-0b2b-4617-83d6-64710834e557 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:31.478212 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cd88f6-0b2b-4617-83d6-64710834e557 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:31.479754 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cd88f6-0b2b-4617-83d6-64710834e557 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:31.493403 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 439/900] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:29:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:31.494599 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd88f6-0b2b-4617-83d6-64710834e557 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:31.531366 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f8404-e19c-45b5-8c4c-969ba079ef67 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:31.567122 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bd7c23-5288-4231-9fd0-4cbf3d43f7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:31.567720 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd7c23-5288-4231-9fd0-4cbf3d43f7ac None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:31.568014 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd7c23-5288-4231-9fd0-4cbf3d43f7ac None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:31.568299 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd7c23-5288-4231-9fd0-4cbf3d43f7ac None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:31.569017 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bd7c23-5288-4231-9fd0-4cbf3d43f7ac None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:31.569375 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bd7c23-5288-4231-9fd0-4cbf3d43f7ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:31.570805 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bd7c23-5288-4231-9fd0-4cbf3d43f7ac None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:31.575995 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 462/901] 10.222.0.36 () {56 vars in 1126 bytes} [Thu May 19 16:29:31 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:31.576803 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bd7c23-5288-4231-9fd0-4cbf3d43f7ac None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:31.585578 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:31.599935 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd88f6-0b2b-4617-83d6-64710834e557 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:31.634868 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a92e30-4c7e-4675-8564-0a6b9e31e7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:31.635530 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a92e30-4c7e-4675-8564-0a6b9e31e7a4 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:31.635812 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a92e30-4c7e-4675-8564-0a6b9e31e7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:31.636091 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a92e30-4c7e-4675-8564-0a6b9e31e7a4 None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:31.636800 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a92e30-4c7e-4675-8564-0a6b9e31e7a4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:31.637107 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a92e30-4c7e-4675-8564-0a6b9e31e7a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:31.638500 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a92e30-4c7e-4675-8564-0a6b9e31e7a4 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:31.643512 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 440/902] 10.222.0.36 () {56 vars in 1114 bytes} [Thu May 19 16:29:31 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:31.644865 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73fba5-06d0-4822-8c9f-4da01b795267 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:31.645174 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73fba5-06d0-4822-8c9f-4da01b795267 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:31.645463 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73fba5-06d0-4822-8c9f-4da01b795267 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:31.646696 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 441/903] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:29:31.647390 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b73fba5-06d0-4822-8c9f-4da01b795267 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:31.677334 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:31.683078 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bd7c23-5288-4231-9fd0-4cbf3d43f7ac None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:31.717292 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3339eb-2d0d-4015-acb7-0c7583d20871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:31.718599 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3339eb-2d0d-4015-acb7-0c7583d20871 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:31.718891 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3339eb-2d0d-4015-acb7-0c7583d20871 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:31.719173 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3339eb-2d0d-4015-acb7-0c7583d20871 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:31.726181 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3339eb-2d0d-4015-acb7-0c7583d20871 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:31.727563 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3339eb-2d0d-4015-acb7-0c7583d20871 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:31.744076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-7e3339eb-2d0d-4015-acb7-0c7583d20871 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:29:31.744076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:31.744076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:31.744076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:31.744076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:31.744076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:31.744076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:31.744076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:31.744076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:31.744076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:31.744076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:31.744076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:31.744076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 16:29:31.744076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 16:29:31.744076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 16:29:31.745091 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 16:29:31.745091 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:31.745091 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:31.745091 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:31.745091 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:31.745091 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:31.745091 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:31.745091 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:31.745091 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:31.745091 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:31.745091 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:31.745091 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:31.745091 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:31.745091 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:31.745091 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:31.745091 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:31.745091 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:31.746033 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 16:29:31.746033 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 16:29:31.746033 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 16:29:31.746033 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 16:29:31.746033 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 16:29:31.746033 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 16:29:31.746033 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:29:31.746033 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:31.746033 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 463/904] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:29:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:31.746531 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3339eb-2d0d-4015-acb7-0c7583d20871 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:31.761212 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b73fba5-06d0-4822-8c9f-4da01b795267 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:31.796044 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3006ee61-32e6-4903-b44a-4d6830c73ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:31.796717 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006ee61-32e6-4903-b44a-4d6830c73ec0 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:31.796999 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006ee61-32e6-4903-b44a-4d6830c73ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:31.797314 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006ee61-32e6-4903-b44a-4d6830c73ec0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:31.798113 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3006ee61-32e6-4903-b44a-4d6830c73ec0 None admin] RBAC: Adding query filter params (interface=public, service_id=22550d86c8d6444392e8f3cc223b43b8, region_id=RegionOne) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:31.798421 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3006ee61-32e6-4903-b44a-4d6830c73ec0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:31.799794 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3006ee61-32e6-4903-b44a-4d6830c73ec0 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:31.807506 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 442/905] 10.222.0.36 () {56 vars in 1251 bytes} [Thu May 19 16:29:31 2022] GET /identity/v3/endpoints?service_id=22550d86c8d6444392e8f3cc223b43b8&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:31.808796 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c97a4-7907-4587-b0c2-6b7f8245146b None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:31.809093 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c97a4-7907-4587-b0c2-6b7f8245146b None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:31.809405 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c97a4-7907-4587-b0c2-6b7f8245146b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:31.816416 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:31.826827 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-d99c97a4-7907-4587-b0c2-6b7f8245146b None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:31.828460 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-d99c97a4-7907-4587-b0c2-6b7f8245146b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:31.851662 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3339eb-2d0d-4015-acb7-0c7583d20871 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:31.885062 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033cf789-b7b3-4462-910e-34b8fe433912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:31.885758 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033cf789-b7b3-4462-910e-34b8fe433912 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:31.886045 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033cf789-b7b3-4462-910e-34b8fe433912 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:31.886335 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033cf789-b7b3-4462-910e-34b8fe433912 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:31.890535 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033cf789-b7b3-4462-910e-34b8fe433912 None admin] RBAC: Adding query filter params (interface=public, service_id=53fd50fe2d7a45e2b997988ca4a86335, region_id=RegionOne) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:31.890535 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033cf789-b7b3-4462-910e-34b8fe433912 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:31.890535 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033cf789-b7b3-4462-910e-34b8fe433912 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:31.895497 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 443/906] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:31 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:29:31.895942 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 464/907] 10.222.0.36 () {56 vars in 1251 bytes} [Thu May 19 16:29:31 2022] GET /identity/v3/endpoints?service_id=53fd50fe2d7a45e2b997988ca4a86335&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:31.896696 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99c97a4-7907-4587-b0c2-6b7f8245146b None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:31.896870 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033cf789-b7b3-4462-910e-34b8fe433912 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:31.923578 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:32.001811 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99c97a4-7907-4587-b0c2-6b7f8245146b None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:32.009672 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033cf789-b7b3-4462-910e-34b8fe433912 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:32.035550 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286570b-0dc2-457a-b839-5bef3ee2e12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:32.036217 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286570b-0dc2-457a-b839-5bef3ee2e12e None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:32.036502 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286570b-0dc2-457a-b839-5bef3ee2e12e None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:32.036788 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286570b-0dc2-457a-b839-5bef3ee2e12e None admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:32.037612 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7286570b-0dc2-457a-b839-5bef3ee2e12e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:32.037926 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7286570b-0dc2-457a-b839-5bef3ee2e12e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:32.039167 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:32.039364 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7286570b-0dc2-457a-b839-5bef3ee2e12e None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:32.045518 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622e32f2-7252-46d2-a095-cb8ec74904ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:32.046706 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e32f2-7252-46d2-a095-cb8ec74904ba None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:32.046986 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e32f2-7252-46d2-a095-cb8ec74904ba None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:32.047261 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e32f2-7252-46d2-a095-cb8ec74904ba None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:32.047960 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622e32f2-7252-46d2-a095-cb8ec74904ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:32.049345 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622e32f2-7252-46d2-a095-cb8ec74904ba None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:32.051635 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 444/908] 10.222.0.36 () {56 vars in 1148 bytes} [Thu May 19 16:29:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:32.052469 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286570b-0dc2-457a-b839-5bef3ee2e12e None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:32.065413 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 465/909] 10.222.0.36 () {56 vars in 1087 bytes} [Thu May 19 16:29:31 2022] GET /identity/v3/services => generated 1268 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:29:32.065413 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622e32f2-7252-46d2-a095-cb8ec74904ba None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:32.150255 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:32.172916 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286570b-0dc2-457a-b839-5bef3ee2e12e None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:32.174435 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622e32f2-7252-46d2-a095-cb8ec74904ba None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:32.210213 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc3291c-6a65-4762-81ee-7cd673f00f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:32.210895 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc3291c-6a65-4762-81ee-7cd673f00f21 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:32.211180 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc3291c-6a65-4762-81ee-7cd673f00f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:32.211459 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc3291c-6a65-4762-81ee-7cd673f00f21 None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:32.212169 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc3291c-6a65-4762-81ee-7cd673f00f21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:32.213240 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e57915-44ee-436d-9b61-c663a37a3c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:32.213602 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc3291c-6a65-4762-81ee-7cd673f00f21 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:32.213916 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e57915-44ee-436d-9b61-c663a37a3c2a None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:32.214202 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e57915-44ee-436d-9b61-c663a37a3c2a None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:32.214479 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e57915-44ee-436d-9b61-c663a37a3c2a None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:32.215350 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e57915-44ee-436d-9b61-c663a37a3c2a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:32.216884 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e57915-44ee-436d-9b61-c663a37a3c2a None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:32.221058 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 445/910] 10.222.0.36 () {56 vars in 1087 bytes} [Thu May 19 16:29:32 2022] GET /identity/v3/services => generated 1268 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:29:32.221978 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc3291c-6a65-4762-81ee-7cd673f00f21 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:32.249376 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 466/911] 10.222.0.36 () {60 vars in 1141 bytes} [Thu May 19 16:29:32 2022] POST /identity/v3/services => generated 241 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:29:32.373088 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc3291c-6a65-4762-81ee-7cd673f00f21 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:32.424914 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c56520-d49f-4d5b-ab8b-ab1b736ec801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:32.425667 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c56520-d49f-4d5b-ab8b-ab1b736ec801 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:32.426018 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c56520-d49f-4d5b-ab8b-ab1b736ec801 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:32.426367 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c56520-d49f-4d5b-ab8b-ab1b736ec801 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:32.427373 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c56520-d49f-4d5b-ab8b-ab1b736ec801 None admin] RBAC: Adding query filter params (role.id=58a72d6ac3ff4e278e79059c4fe2ec5a, scope.project.id=631c76794a124d6da22c626e590e4352, user.id=a30dc1dd2b00463bb3c7957864c56043) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:32.427761 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c56520-d49f-4d5b-ab8b-ab1b736ec801 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:32.429395 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c56520-d49f-4d5b-ab8b-ab1b736ec801 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:32.437025 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 446/912] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:29:32 2022] GET /identity/v3/role_assignments?user.id=a30dc1dd2b00463bb3c7957864c56043&scope.project.id=631c76794a124d6da22c626e590e4352&role.id=58a72d6ac3ff4e278e79059c4fe2ec5a => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:34.390535 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db1a56-1d23-4865-b286-d6d2b66a1485 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:34.391546 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db1a56-1d23-4865-b286-d6d2b66a1485 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:34.392044 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db1a56-1d23-4865-b286-d6d2b66a1485 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:34.394077 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 467/913] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:29:34.400524 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b6643-be60-462b-b5ca-8f298516c646 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:34.401517 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b6643-be60-462b-b5ca-8f298516c646 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:34.402013 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b6643-be60-462b-b5ca-8f298516c646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:34.414796 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:34.421097 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-a55b6643-be60-462b-b5ca-8f298516c646 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:34.422379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-a55b6643-be60-462b-b5ca-8f298516c646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:34.501071 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 447/914] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:34 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:29:34.524670 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4db1a56-1d23-4865-b286-d6d2b66a1485 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:34.573668 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc31f9-e23a-46a5-bc48-bd6c997e86fa None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:34.573996 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc31f9-e23a-46a5-bc48-bd6c997e86fa None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:34.574239 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc31f9-e23a-46a5-bc48-bd6c997e86fa None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:34.576047 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 448/915] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 16:29:34.582316 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a21327e-5d60-4eee-b5cc-a325cfd5f360 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:34.582619 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a21327e-5d60-4eee-b5cc-a325cfd5f360 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:34.582878 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a21327e-5d60-4eee-b5cc-a325cfd5f360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:34.598613 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:34.604212 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-0a21327e-5d60-4eee-b5cc-a325cfd5f360 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:34.609227 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-0a21327e-5d60-4eee-b5cc-a325cfd5f360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:34.653277 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4db1a56-1d23-4865-b286-d6d2b66a1485 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:34.695278 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d44480b-5ff8-4db8-8570-4c5fcf288c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:34.696235 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44480b-5ff8-4db8-8570-4c5fcf288c98 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:34.696526 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44480b-5ff8-4db8-8570-4c5fcf288c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:34.696806 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44480b-5ff8-4db8-8570-4c5fcf288c98 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:34.697513 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d44480b-5ff8-4db8-8570-4c5fcf288c98 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:34.697623 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 449/916] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:34 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 16:29:34.699485 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d44480b-5ff8-4db8-8570-4c5fcf288c98 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:34.700835 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af45fb-ed40-4be2-bc26-ca44a339c78b None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:34.701134 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af45fb-ed40-4be2-bc26-ca44a339c78b None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:34.701465 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af45fb-ed40-4be2-bc26-ca44a339c78b None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:34.703058 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 450/917] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:29:34.704172 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07043bb2-9ee2-461e-a458-567c62cc74b9 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:34.704461 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07043bb2-9ee2-461e-a458-567c62cc74b9 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:34.704741 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07043bb2-9ee2-461e-a458-567c62cc74b9 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:34.706016 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 451/918] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:29:34.711163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07c530-fd9d-42ce-8a56-a9565bdf143e None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:34.711449 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07c530-fd9d-42ce-8a56-a9565bdf143e None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:34.711734 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07c530-fd9d-42ce-8a56-a9565bdf143e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:34.714790 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-3d44480b-5ff8-4db8-8570-4c5fcf288c98 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 16:29:34.714790 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:34.714790 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 16:29:34.714790 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 16:29:34.714790 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 16:29:34.714790 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 16:29:34.714790 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 16:29:34.714790 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:29:34.714790 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 16:29:34.714790 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:34.714790 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 16:29:34.714790 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:34.714790 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:34.714790 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:34.714790 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:34.714790 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:34.715867 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:34.715867 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:34.715867 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:34.715867 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:34.715867 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:34.715867 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:34.715867 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:34.715867 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 16:29:34.715867 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 16:29:34.715867 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 16:29:34.715867 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 16:29:34.715867 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:34.715867 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:34.715867 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:34.715867 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:34.715867 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:34.716711 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:34.716711 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:34.716711 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:34.716711 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:34.716711 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:34.716711 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:34.716711 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:34.716711 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:34.716711 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:34.716711 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:34.716711 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:34.716711 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 16:29:34.716711 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:29:34.716711 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 16:29:34.716711 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:34.717537 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 468/919] 10.222.0.36 () {56 vars in 1132 bytes} [Thu May 19 16:29:34 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:34.719143 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c68406-eb14-4a03-825f-557eba586d3b None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:34.719430 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c68406-eb14-4a03-825f-557eba586d3b None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:34.719718 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c68406-eb14-4a03-825f-557eba586d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:34.735415 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-9b07c530-fd9d-42ce-8a56-a9565bdf143e None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:34.736622 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-9b07c530-fd9d-42ce-8a56-a9565bdf143e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:34.737090 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-b5c68406-eb14-4a03-825f-557eba586d3b None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:34.738295 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b5c68406-eb14-4a03-825f-557eba586d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:34.801134 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:34.809575 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 452/920] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:34 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:29:34.809800 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 469/921] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:34 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:29:34.810614 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07c530-fd9d-42ce-8a56-a9565bdf143e None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:34.810992 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c68406-eb14-4a03-825f-557eba586d3b None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:34.878087 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:34.929096 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07c530-fd9d-42ce-8a56-a9565bdf143e None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:34.932368 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c68406-eb14-4a03-825f-557eba586d3b None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:34.963442 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a002ae3-4c95-45d9-9bc5-4b0298fd2824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:34.964296 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a002ae3-4c95-45d9-9bc5-4b0298fd2824 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:34.964537 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a002ae3-4c95-45d9-9bc5-4b0298fd2824 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:34.964814 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a002ae3-4c95-45d9-9bc5-4b0298fd2824 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:34.965507 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a002ae3-4c95-45d9-9bc5-4b0298fd2824 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:34.966883 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12567fac-aedc-4b01-9e99-d96b150b7d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:34.966990 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a002ae3-4c95-45d9-9bc5-4b0298fd2824 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:34.967556 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12567fac-aedc-4b01-9e99-d96b150b7d1c None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:34.967893 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12567fac-aedc-4b01-9e99-d96b150b7d1c None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:34.968681 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12567fac-aedc-4b01-9e99-d96b150b7d1c None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:34.969430 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12567fac-aedc-4b01-9e99-d96b150b7d1c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:34.969744 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12567fac-aedc-4b01-9e99-d96b150b7d1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:34.971118 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12567fac-aedc-4b01-9e99-d96b150b7d1c None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:34.975933 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 470/922] 10.222.0.36 () {56 vars in 1126 bytes} [Thu May 19 16:29:34 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:34.976713 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12567fac-aedc-4b01-9e99-d96b150b7d1c None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:34.979455 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-5a002ae3-4c95-45d9-9bc5-4b0298fd2824 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 16:29:34.979455 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:34.979455 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 16:29:34.979455 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 16:29:34.979455 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 16:29:34.979455 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 16:29:34.979455 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 16:29:34.979455 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:29:34.979455 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 16:29:34.979455 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:34.979455 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 16:29:34.979455 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:34.979455 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:34.979455 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:34.979455 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:34.979455 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:34.979455 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:34.980533 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:34.980533 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:34.980533 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:34.980533 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:34.980533 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:34.980533 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:34.980533 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 16:29:34.980533 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 16:29:34.980533 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 16:29:34.980533 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 16:29:34.980533 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:34.980533 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:34.980533 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:34.980533 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:34.980533 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:34.980533 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:34.981985 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:34.981985 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:34.981985 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:34.981985 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:34.981985 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:34.981985 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:34.981985 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:34.981985 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:34.981985 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:34.981985 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:34.981985 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 16:29:34.981985 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:29:34.981985 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 16:29:34.981985 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:34.981985 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 453/923] 10.222.0.36 () {56 vars in 1114 bytes} [Thu May 19 16:29:34 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:34.983051 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a002ae3-4c95-45d9-9bc5-4b0298fd2824 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:35.042160 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:35.084512 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a002ae3-4c95-45d9-9bc5-4b0298fd2824 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:35.084730 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12567fac-aedc-4b01-9e99-d96b150b7d1c None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:35.099009 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:35.118320 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f798331-7dce-4687-944d-57f865e93bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:35.119143 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f798331-7dce-4687-944d-57f865e93bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:35.119430 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f798331-7dce-4687-944d-57f865e93bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:35.119703 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f798331-7dce-4687-944d-57f865e93bc3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:35.120485 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd217ca-4330-45cd-aa29-cd9dec7ec752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:35.121321 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd217ca-4330-45cd-aa29-cd9dec7ec752 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:35.121612 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd217ca-4330-45cd-aa29-cd9dec7ec752 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:35.121889 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd217ca-4330-45cd-aa29-cd9dec7ec752 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:35.122551 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd217ca-4330-45cd-aa29-cd9dec7ec752 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:35.123944 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd217ca-4330-45cd-aa29-cd9dec7ec752 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:35.130917 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f798331-7dce-4687-944d-57f865e93bc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:29:35.136694 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-4bd217ca-4330-45cd-aa29-cd9dec7ec752 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 16:29:35.136694 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:35.136694 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 16:29:35.136694 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 16:29:35.136694 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 16:29:35.136694 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 16:29:35.136694 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 16:29:35.136694 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:29:35.136694 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 16:29:35.136694 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:35.136694 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 16:29:35.136694 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:35.136694 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:35.136694 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:35.136694 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:35.136694 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:35.136694 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:35.137727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:35.137727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:35.137727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:35.137727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:35.137727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:35.137727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:35.137727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 16:29:35.137727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 16:29:35.137727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 16:29:35.137727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 16:29:35.137727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:35.137727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:35.137727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:35.137727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:35.137727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:35.137727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:35.138577 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:35.138577 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:35.138577 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:35.138577 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:35.138577 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:35.138577 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:35.138577 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:35.138577 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:35.138577 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:35.138577 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:35.138577 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 16:29:35.138577 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:29:35.138577 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 16:29:35.138577 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:35.138577 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 471/924] 10.222.0.36 () {56 vars in 1126 bytes} [Thu May 19 16:29:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:35.139375 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd217ca-4330-45cd-aa29-cd9dec7ec752 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:35.139454 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-4f798331-7dce-4687-944d-57f865e93bc3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 16:29:35.139454 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:35.139454 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:35.139454 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:35.139454 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:35.139454 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:35.139454 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:35.139454 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:35.139454 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:35.139454 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:35.139454 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:35.139454 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:35.139454 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:29:35.139454 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:29:35.139454 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:29:35.139454 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:29:35.140883 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:29:35.140883 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:29:35.140883 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:29:35.140883 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:29:35.140883 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:35.140883 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:35.140883 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:35.140883 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:35.140883 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:35.140883 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:35.140883 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:35.140883 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:35.140883 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:35.140883 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:35.140883 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:35.140883 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:35.141797 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:35.141797 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:35.141797 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:35.141797 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:35.141797 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:29:35.141797 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:29:35.141797 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:29:35.141797 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:29:35.141797 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:29:35.141797 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:29:35.141797 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 16:29:35.141797 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:35.141797 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 454/925] 10.222.0.36 () {56 vars in 1120 bytes} [Thu May 19 16:29:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:35.142497 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f798331-7dce-4687-944d-57f865e93bc3 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:35.242542 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f798331-7dce-4687-944d-57f865e93bc3 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:35.243488 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd217ca-4330-45cd-aa29-cd9dec7ec752 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:35.262052 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:35.277351 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d744074b-9882-4640-8a37-103894afde1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:35.277930 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8922f9-f67b-4477-b89b-49c6d7cca0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:35.278054 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744074b-9882-4640-8a37-103894afde1b None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:35.278287 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744074b-9882-4640-8a37-103894afde1b None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:35.278564 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744074b-9882-4640-8a37-103894afde1b None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:35.278674 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8922f9-f67b-4477-b89b-49c6d7cca0f5 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:35.278854 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8922f9-f67b-4477-b89b-49c6d7cca0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:35.279128 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8922f9-f67b-4477-b89b-49c6d7cca0f5 None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:35.279281 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d744074b-9882-4640-8a37-103894afde1b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:35.279587 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d744074b-9882-4640-8a37-103894afde1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:35.279833 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8922f9-f67b-4477-b89b-49c6d7cca0f5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:35.280140 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8922f9-f67b-4477-b89b-49c6d7cca0f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:35.280986 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d744074b-9882-4640-8a37-103894afde1b None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:35.281683 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8922f9-f67b-4477-b89b-49c6d7cca0f5 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:35.285792 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 455/926] 10.222.0.36 () {56 vars in 1114 bytes} [Thu May 19 16:29:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:35.291006 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d744074b-9882-4640-8a37-103894afde1b None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:35.291154 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 472/927] 10.222.0.36 () {56 vars in 1126 bytes} [Thu May 19 16:29:35 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:35.291154 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8922f9-f67b-4477-b89b-49c6d7cca0f5 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:35.332303 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:35.393399 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8922f9-f67b-4477-b89b-49c6d7cca0f5 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:35.394335 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d744074b-9882-4640-8a37-103894afde1b None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:35.426968 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3801311-6ae1-442a-b96f-c7a38628b94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:35.427626 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3801311-6ae1-442a-b96f-c7a38628b94f None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:35.427909 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3801311-6ae1-442a-b96f-c7a38628b94f None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:35.428188 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3801311-6ae1-442a-b96f-c7a38628b94f None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:35.428342 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a9d40-5eb7-4f3e-8df1-6cd8316798ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:35.428906 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3801311-6ae1-442a-b96f-c7a38628b94f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:35.428999 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a9d40-5eb7-4f3e-8df1-6cd8316798ec None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:35.429245 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3801311-6ae1-442a-b96f-c7a38628b94f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:35.429340 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a9d40-5eb7-4f3e-8df1-6cd8316798ec None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:35.429573 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a9d40-5eb7-4f3e-8df1-6cd8316798ec None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:35.430316 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9a9d40-5eb7-4f3e-8df1-6cd8316798ec None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:35.430619 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3801311-6ae1-442a-b96f-c7a38628b94f None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:35.430709 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9a9d40-5eb7-4f3e-8df1-6cd8316798ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:35.432014 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9a9d40-5eb7-4f3e-8df1-6cd8316798ec None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:35.435370 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 456/928] 10.222.0.36 () {56 vars in 1122 bytes} [Thu May 19 16:29:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:35.436163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3801311-6ae1-442a-b96f-c7a38628b94f None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:35.440939 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 473/929] 10.222.0.36 () {56 vars in 1115 bytes} [Thu May 19 16:29:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:35.441790 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a9d40-5eb7-4f3e-8df1-6cd8316798ec None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:35.492637 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:35.544404 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3801311-6ae1-442a-b96f-c7a38628b94f None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:35.553181 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a9d40-5eb7-4f3e-8df1-6cd8316798ec None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:35.554959 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:35.582612 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d31c3bc-ca48-42a7-b894-068c40823baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:35.583301 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d31c3bc-ca48-42a7-b894-068c40823baf None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:35.583589 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d31c3bc-ca48-42a7-b894-068c40823baf None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:35.583881 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d31c3bc-ca48-42a7-b894-068c40823baf None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:35.584616 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d31c3bc-ca48-42a7-b894-068c40823baf None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:35.584934 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d31c3bc-ca48-42a7-b894-068c40823baf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:35.586384 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d31c3bc-ca48-42a7-b894-068c40823baf None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:35.591632 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b914168d-fd14-4881-9374-cec2a5816172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:35.592163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 457/930] 10.222.0.36 () {56 vars in 1114 bytes} [Thu May 19 16:29:35 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:35.592313 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914168d-fd14-4881-9374-cec2a5816172 None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:35.592598 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914168d-fd14-4881-9374-cec2a5816172 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:35.592876 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914168d-fd14-4881-9374-cec2a5816172 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:35.593002 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d31c3bc-ca48-42a7-b894-068c40823baf None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:35.593676 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b914168d-fd14-4881-9374-cec2a5816172 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:35.595099 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b914168d-fd14-4881-9374-cec2a5816172 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:35.616495 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 474/931] 10.222.0.36 () {60 vars in 1144 bytes} [Thu May 19 16:29:35 2022] POST /identity/v3/endpoints => generated 347 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:29:35.617398 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b914168d-fd14-4881-9374-cec2a5816172 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:35.725701 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d31c3bc-ca48-42a7-b894-068c40823baf None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:35.745026 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:35.753288 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b914168d-fd14-4881-9374-cec2a5816172 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:35.771325 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6174d2ac-4611-4e6d-8b5e-f66833a80828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:35.772018 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174d2ac-4611-4e6d-8b5e-f66833a80828 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:35.772303 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174d2ac-4611-4e6d-8b5e-f66833a80828 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:35.772580 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174d2ac-4611-4e6d-8b5e-f66833a80828 None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:35.773420 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6174d2ac-4611-4e6d-8b5e-f66833a80828 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:35.773731 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6174d2ac-4611-4e6d-8b5e-f66833a80828 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:35.775124 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6174d2ac-4611-4e6d-8b5e-f66833a80828 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:35.785284 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 458/932] 10.222.0.36 () {56 vars in 1122 bytes} [Thu May 19 16:29:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:35.785284 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6174d2ac-4611-4e6d-8b5e-f66833a80828 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:35.796666 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aafa72-90e4-4433-b776-53074b3eacfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:35.797668 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aafa72-90e4-4433-b776-53074b3eacfe None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:35.797953 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aafa72-90e4-4433-b776-53074b3eacfe None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:35.798233 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aafa72-90e4-4433-b776-53074b3eacfe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:35.813133 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aafa72-90e4-4433-b776-53074b3eacfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:35.815009 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aafa72-90e4-4433-b776-53074b3eacfe None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:35.836399 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-29aafa72-90e4-4433-b776-53074b3eacfe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:35.836399 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:35.836399 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:35.836399 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:35.836399 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:35.836399 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:35.836399 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:35.836399 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:35.836399 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:35.836399 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:35.836399 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:35.836399 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:35.836399 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:29:35.836399 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:29:35.836399 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:29:35.837472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:29:35.837472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:35.837472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:35.837472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:35.837472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:35.837472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:35.837472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:35.837472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:35.837472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:35.837472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:35.837472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:35.837472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:35.837472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:35.837472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:35.837472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:35.837472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:35.837472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:35.838358 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:29:35.838358 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:29:35.838358 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:35.838358 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:35.838358 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 475/933] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:29:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:35.839015 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aafa72-90e4-4433-b776-53074b3eacfe None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:35.848722 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:35.904774 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6174d2ac-4611-4e6d-8b5e-f66833a80828 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:35.944645 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd6849-7c07-4c42-89ee-0889a6192670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:35.945349 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd6849-7c07-4c42-89ee-0889a6192670 None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:35.945642 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd6849-7c07-4c42-89ee-0889a6192670 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:35.945925 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd6849-7c07-4c42-89ee-0889a6192670 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:35.946682 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedd6849-7c07-4c42-89ee-0889a6192670 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:35.948064 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedd6849-7c07-4c42-89ee-0889a6192670 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:35.952041 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aafa72-90e4-4433-b776-53074b3eacfe None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:35.968726 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 459/934] 10.222.0.36 () {60 vars in 1144 bytes} [Thu May 19 16:29:35 2022] POST /identity/v3/endpoints => generated 346 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:29:35.969606 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd6849-7c07-4c42-89ee-0889a6192670 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:36.008516 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:36.016544 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a372baff-55a7-4e9a-9015-db561e41e8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:36.017676 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372baff-55a7-4e9a-9015-db561e41e8ec None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:36.018165 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372baff-55a7-4e9a-9015-db561e41e8ec None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:36.018640 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372baff-55a7-4e9a-9015-db561e41e8ec None admin] PATH_INFO: `/v3/endpoints` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:36.019921 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a372baff-55a7-4e9a-9015-db561e41e8ec None admin] RBAC: Adding query filter params (interface=public, service_id=617cce2fd363454a97312ee4379d57c4, region_id=RegionOne) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:36.021076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a372baff-55a7-4e9a-9015-db561e41e8ec None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:36.023481 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a372baff-55a7-4e9a-9015-db561e41e8ec None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:36.037109 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 476/935] 10.222.0.36 () {56 vars in 1251 bytes} [Thu May 19 16:29:35 2022] GET /identity/v3/endpoints?service_id=617cce2fd363454a97312ee4379d57c4&interface=public®ion_id=RegionOne => generated 194 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:36.041316 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a372baff-55a7-4e9a-9015-db561e41e8ec None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:36.114553 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd6849-7c07-4c42-89ee-0889a6192670 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:36.121387 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:36.164633 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a372baff-55a7-4e9a-9015-db561e41e8ec None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:36.168812 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3946131d-ddb8-46d1-8fbd-c2f35dd517e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:36.169950 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946131d-ddb8-46d1-8fbd-c2f35dd517e5 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:36.170441 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946131d-ddb8-46d1-8fbd-c2f35dd517e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:36.170982 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946131d-ddb8-46d1-8fbd-c2f35dd517e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:36.172257 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3946131d-ddb8-46d1-8fbd-c2f35dd517e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:36.172568 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3946131d-ddb8-46d1-8fbd-c2f35dd517e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:36.174099 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3946131d-ddb8-46d1-8fbd-c2f35dd517e5 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:36.187829 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 460/936] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:29:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:36.195560 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3946131d-ddb8-46d1-8fbd-c2f35dd517e5 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:36.206491 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9022e3e-0cda-4f15-abfb-6b8d356b6e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:36.207178 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9022e3e-0cda-4f15-abfb-6b8d356b6e3c None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:36.207469 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9022e3e-0cda-4f15-abfb-6b8d356b6e3c None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:36.207750 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9022e3e-0cda-4f15-abfb-6b8d356b6e3c None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:36.208450 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9022e3e-0cda-4f15-abfb-6b8d356b6e3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:36.209887 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9022e3e-0cda-4f15-abfb-6b8d356b6e3c None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:36.216548 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 477/937] 10.222.0.36 () {56 vars in 1087 bytes} [Thu May 19 16:29:36 2022] GET /identity/v3/services => generated 1497 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:29:36.314049 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:36.325131 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3946131d-ddb8-46d1-8fbd-c2f35dd517e5 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:36.367985 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da48dea-a3c3-4abf-ae28-a034e0043909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:36.369306 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da48dea-a3c3-4abf-ae28-a034e0043909 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:36.369773 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da48dea-a3c3-4abf-ae28-a034e0043909 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:36.370194 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da48dea-a3c3-4abf-ae28-a034e0043909 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:36.378999 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da48dea-a3c3-4abf-ae28-a034e0043909 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:36.380767 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da48dea-a3c3-4abf-ae28-a034e0043909 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:36.398505 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-1da48dea-a3c3-4abf-ae28-a034e0043909 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 16:29:36.398505 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:36.398505 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:36.398505 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:36.398505 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:36.398505 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:36.398505 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:36.398505 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:36.398505 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:36.398505 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:36.398505 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:36.398505 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:36.398505 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 16:29:36.398505 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 16:29:36.398505 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 16:29:36.398505 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 16:29:36.399752 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:36.399752 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:36.399752 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 16:29:36.399752 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:29:36.399752 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 16:29:36.399752 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:29:36.399752 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:36.399752 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:36.399752 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:36.399752 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:36.399752 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:36.399752 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:36.399752 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:36.399752 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:36.399752 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:36.399752 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:36.399752 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:36.399752 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:36.400929 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:36.400929 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:36.400929 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 16:29:36.400929 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 16:29:36.400929 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 16:29:36.400929 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 16:29:36.400929 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 16:29:36.400929 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 16:29:36.400929 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 16:29:36.400929 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:36.403364 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 461/938] 10.222.0.36 () {56 vars in 1117 bytes} [Thu May 19 16:29:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 16:29:36.409999 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9022e3e-0cda-4f15-abfb-6b8d356b6e3c None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:36.521697 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9022e3e-0cda-4f15-abfb-6b8d356b6e3c None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:36.553064 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:36.560237 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a315f120-55ec-42db-97cc-6dddcb28e7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:36.560917 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a315f120-55ec-42db-97cc-6dddcb28e7e4 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:36.561225 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a315f120-55ec-42db-97cc-6dddcb28e7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:36.561538 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a315f120-55ec-42db-97cc-6dddcb28e7e4 None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:36.562551 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a315f120-55ec-42db-97cc-6dddcb28e7e4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:36.562861 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a315f120-55ec-42db-97cc-6dddcb28e7e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:36.564374 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a315f120-55ec-42db-97cc-6dddcb28e7e4 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:36.583718 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 478/939] 10.222.0.36 () {56 vars in 1149 bytes} [Thu May 19 16:29:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:36.589249 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da48dea-a3c3-4abf-ae28-a034e0043909 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:36.703591 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da48dea-a3c3-4abf-ae28-a034e0043909 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:36.739532 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a13105-83b0-459d-b0b5-cd27790c6699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:36.740611 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a13105-83b0-459d-b0b5-cd27790c6699 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:36.741050 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a13105-83b0-459d-b0b5-cd27790c6699 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:36.741498 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a13105-83b0-459d-b0b5-cd27790c6699 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:36.753433 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a13105-83b0-459d-b0b5-cd27790c6699 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:36.755168 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a13105-83b0-459d-b0b5-cd27790c6699 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:36.775708 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-d2a13105-83b0-459d-b0b5-cd27790c6699 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:36.775708 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:36.775708 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:36.775708 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:36.775708 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:36.775708 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:36.775708 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:36.775708 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:36.775708 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:36.775708 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:36.775708 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:36.775708 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:36.775708 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:29:36.775708 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:29:36.775708 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:29:36.776793 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:29:36.776793 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:36.776793 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:36.776793 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:36.776793 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:36.776793 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:36.776793 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:36.776793 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:36.776793 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:36.776793 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:36.776793 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:36.776793 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:36.776793 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:36.776793 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:36.776793 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:36.776793 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:36.776793 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:36.777830 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:29:36.777830 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:29:36.777830 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:36.777830 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:36.779851 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 462/940] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:29:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:36.786318 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a315f120-55ec-42db-97cc-6dddcb28e7e4 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:36.898699 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a315f120-55ec-42db-97cc-6dddcb28e7e4 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:36.934215 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da687067-6afb-4828-a945-53987bbf45e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:36.934888 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da687067-6afb-4828-a945-53987bbf45e6 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:36.935171 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da687067-6afb-4828-a945-53987bbf45e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:36.935448 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da687067-6afb-4828-a945-53987bbf45e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:36.936183 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da687067-6afb-4828-a945-53987bbf45e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:36.937010 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da687067-6afb-4828-a945-53987bbf45e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:36.938438 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da687067-6afb-4828-a945-53987bbf45e6 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:36.950744 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 479/941] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:29:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:36.956874 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a13105-83b0-459d-b0b5-cd27790c6699 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:36.961011 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:37.071377 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a13105-83b0-459d-b0b5-cd27790c6699 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:37.108907 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d888b3-c5cc-4eb2-8526-f9e9bd9b30b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:37.110130 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d888b3-c5cc-4eb2-8526-f9e9bd9b30b1 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:37.110653 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d888b3-c5cc-4eb2-8526-f9e9bd9b30b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:37.111157 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d888b3-c5cc-4eb2-8526-f9e9bd9b30b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:37.118596 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d888b3-c5cc-4eb2-8526-f9e9bd9b30b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:37.120314 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d888b3-c5cc-4eb2-8526-f9e9bd9b30b1 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:37.134880 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-87d888b3-c5cc-4eb2-8526-f9e9bd9b30b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:29:37.134880 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:37.134880 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:37.134880 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:37.134880 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:37.134880 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:37.134880 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:37.134880 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:37.134880 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:37.134880 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:37.134880 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:37.134880 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:37.134880 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 16:29:37.134880 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 16:29:37.134880 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 16:29:37.135812 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 16:29:37.135812 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:37.135812 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:37.135812 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:37.135812 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:37.135812 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:37.135812 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:37.135812 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:37.135812 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:37.135812 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:37.135812 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:37.135812 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:37.135812 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:37.135812 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:37.135812 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:37.135812 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:37.135812 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:37.136713 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 16:29:37.136713 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 16:29:37.136713 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 16:29:37.136713 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 16:29:37.136713 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 16:29:37.136713 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 16:29:37.136713 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:29:37.136713 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:37.139183 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 463/942] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:29:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:37.146277 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da687067-6afb-4828-a945-53987bbf45e6 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:37.200203 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:37.256630 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da687067-6afb-4828-a945-53987bbf45e6 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:37.292552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e6205-f5a7-4d67-8159-8e07df3846ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:37.293253 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e6205-f5a7-4d67-8159-8e07df3846ca None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:37.293539 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e6205-f5a7-4d67-8159-8e07df3846ca None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:37.293812 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e6205-f5a7-4d67-8159-8e07df3846ca None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:37.294603 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6e6205-f5a7-4d67-8159-8e07df3846ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:37.294914 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6e6205-f5a7-4d67-8159-8e07df3846ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:37.296403 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6e6205-f5a7-4d67-8159-8e07df3846ca None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:37.309529 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 480/943] 10.222.0.36 () {56 vars in 1148 bytes} [Thu May 19 16:29:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:37.315363 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d888b3-c5cc-4eb2-8526-f9e9bd9b30b1 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:37.382988 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:37.428844 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d888b3-c5cc-4eb2-8526-f9e9bd9b30b1 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:37.466662 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862afbec-8abc-4694-abeb-9ddde7917094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:37.468537 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862afbec-8abc-4694-abeb-9ddde7917094 None admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:37.469058 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862afbec-8abc-4694-abeb-9ddde7917094 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:37.469566 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862afbec-8abc-4694-abeb-9ddde7917094 None admin] PATH_INFO: `/v3/projects/631c76794a124d6da22c626e590e4352/users/a30dc1dd2b00463bb3c7957864c56043/roles/58a72d6ac3ff4e278e79059c4fe2ec5a` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:37.472132 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862afbec-8abc-4694-abeb-9ddde7917094 None admin] RBAC: Authorizing `identity:create_grant(project_id=631c76794a124d6da22c626e590e4352, user_id=a30dc1dd2b00463bb3c7957864c56043, role_id=58a72d6ac3ff4e278e79059c4fe2ec5a)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:37.473835 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862afbec-8abc-4694-abeb-9ddde7917094 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:37.489291 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 464/944] 10.222.0.36 () {58 vars in 1439 bytes} [Thu May 19 16:29:37 2022] PUT /identity/v3/projects/631c76794a124d6da22c626e590e4352/users/a30dc1dd2b00463bb3c7957864c56043/roles/58a72d6ac3ff4e278e79059c4fe2ec5a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:29:37.930385 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd8f06-3f26-4e89-9698-60a4b2021270 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:37.930905 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd8f06-3f26-4e89-9698-60a4b2021270 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:37.930985 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd8f06-3f26-4e89-9698-60a4b2021270 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:37.932644 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 481/945] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:29:37.938537 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2cca24-c0b2-4aa1-93d0-a7b6e7e70bed None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:37.938940 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2cca24-c0b2-4aa1-93d0-a7b6e7e70bed None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:37.939110 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2cca24-c0b2-4aa1-93d0-a7b6e7e70bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:37.956891 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-cb2cca24-c0b2-4aa1-93d0-a7b6e7e70bed None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:37.958305 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-cb2cca24-c0b2-4aa1-93d0-a7b6e7e70bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:38.034444 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 465/946] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 16:29:38.056289 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fd8f06-3f26-4e89-9698-60a4b2021270 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:38.166293 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fd8f06-3f26-4e89-9698-60a4b2021270 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:38.178222 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:38.206012 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6331724-cc05-44a7-b43a-78ff9cf8950e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:38.206819 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6331724-cc05-44a7-b43a-78ff9cf8950e None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:38.207104 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6331724-cc05-44a7-b43a-78ff9cf8950e None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:38.207382 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6331724-cc05-44a7-b43a-78ff9cf8950e None admin] PATH_INFO: `/v3/services/compute` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:38.208045 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6331724-cc05-44a7-b43a-78ff9cf8950e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:38.209457 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6331724-cc05-44a7-b43a-78ff9cf8950e None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:38.221587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-c6331724-cc05-44a7-b43a-78ff9cf8950e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 16:29:38.221587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:38.221587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 16:29:38.221587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 16:29:38.221587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 16:29:38.221587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 16:29:38.221587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 16:29:38.221587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:29:38.221587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 16:29:38.221587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:38.221587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 16:29:38.221587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:38.221587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:38.221587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:38.221587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:38.221587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:38.221587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:38.222621 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:38.222621 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:38.222621 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:38.222621 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:38.222621 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:38.222621 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:38.222621 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 16:29:38.222621 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 16:29:38.222621 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 16:29:38.222621 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 16:29:38.222621 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:38.222621 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:38.222621 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:38.222621 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:38.222621 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:38.222621 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:38.223699 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:38.223699 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:38.223699 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:38.223699 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:38.223699 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:38.223699 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:38.223699 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:38.223699 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:38.223699 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:38.223699 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:38.223699 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 16:29:38.223699 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:29:38.223699 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 16:29:38.223699 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:38.223699 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 482/947] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:29:38 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:38.231047 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2cca24-c0b2-4aa1-93d0-a7b6e7e70bed None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:38.291898 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:38.339382 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2cca24-c0b2-4aa1-93d0-a7b6e7e70bed None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:38.354813 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2255fc16-7c3c-4d21-847e-2aaa25eb1904 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:38.355449 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2255fc16-7c3c-4d21-847e-2aaa25eb1904 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:38.355942 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2255fc16-7c3c-4d21-847e-2aaa25eb1904 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:38.357461 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 483/948] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:29:38.363494 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95cd38e-0672-4cba-8f2f-52961430d5f5 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:38.363781 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95cd38e-0672-4cba-8f2f-52961430d5f5 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:38.364062 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95cd38e-0672-4cba-8f2f-52961430d5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:38.380723 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c7e887-4967-44bd-86f3-13156761c4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:38.381451 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7e887-4967-44bd-86f3-13156761c4fd None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:38.381734 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7e887-4967-44bd-86f3-13156761c4fd None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:38.381837 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-d95cd38e-0672-4cba-8f2f-52961430d5f5 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:38.382016 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7e887-4967-44bd-86f3-13156761c4fd None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:38.382748 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c7e887-4967-44bd-86f3-13156761c4fd None admin] RBAC: Adding query filter params (name=compute) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:38.382940 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d95cd38e-0672-4cba-8f2f-52961430d5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:38.383072 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c7e887-4967-44bd-86f3-13156761c4fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:38.384420 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c7e887-4967-44bd-86f3-13156761c4fd None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:38.394897 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 466/949] 10.222.0.36 () {56 vars in 1112 bytes} [Thu May 19 16:29:38 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:38.396398 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c7e887-4967-44bd-86f3-13156761c4fd None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:38.478206 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 484/950] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 16:29:38.502235 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95cd38e-0672-4cba-8f2f-52961430d5f5 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:38.513097 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c7e887-4967-44bd-86f3-13156761c4fd None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:38.532632 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:38.552724 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b051064f-e0ad-4b46-8782-c9e3f915860c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:38.553425 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051064f-e0ad-4b46-8782-c9e3f915860c None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:38.553709 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051064f-e0ad-4b46-8782-c9e3f915860c None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:38.553986 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051064f-e0ad-4b46-8782-c9e3f915860c None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:38.554705 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b051064f-e0ad-4b46-8782-c9e3f915860c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:38.555015 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b051064f-e0ad-4b46-8782-c9e3f915860c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:38.556392 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b051064f-e0ad-4b46-8782-c9e3f915860c None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:38.561428 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 467/951] 10.222.0.36 () {56 vars in 1112 bytes} [Thu May 19 16:29:38 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:29:38.618921 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95cd38e-0672-4cba-8f2f-52961430d5f5 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:38.621523 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:38.661431 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa182b-7f11-4311-a170-24108b69e181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:38.662140 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa182b-7f11-4311-a170-24108b69e181 None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:38.662437 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa182b-7f11-4311-a170-24108b69e181 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:38.662719 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa182b-7f11-4311-a170-24108b69e181 None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:38.663118 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc5f74-8ebf-48a8-be03-e33389fb7bdc None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:38.663471 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fa182b-7f11-4311-a170-24108b69e181 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:38.663854 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc5f74-8ebf-48a8-be03-e33389fb7bdc None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:38.664356 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc5f74-8ebf-48a8-be03-e33389fb7bdc None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:38.665014 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fa182b-7f11-4311-a170-24108b69e181 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:38.666553 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: WARNING keystone.server.flask.common [None req-24fa182b-7f11-4311-a170-24108b69e181 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 19 16:29:38.667037 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 468/952] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 16:29:38.673963 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae3734-8804-47da-aa27-804bc918ae37 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:38.674650 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae3734-8804-47da-aa27-804bc918ae37 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:38.675135 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae3734-8804-47da-aa27-804bc918ae37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:38.688398 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.sql.core [None req-24fa182b-7f11-4311-a170-24108b69e181 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 16:29:38.688398 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 16:29:38.688398 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [parameters: {'id': 'e536d662b3fc43ab8b4e09f1779f9fa8', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 16:29:38.688398 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80466) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 16:29:38.694656 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-abae3734-8804-47da-aa27-804bc918ae37 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:38.696261 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-abae3734-8804-47da-aa27-804bc918ae37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:38.706320 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-24fa182b-7f11-4311-a170-24108b69e181 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e536d662b3fc43ab8b4e09f1779f9fa8.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e536d662b3fc43ab8b4e09f1779f9fa8. May 19 16:29:38.706320 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:38.706320 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 16:29:38.706320 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 16:29:38.706320 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 16:29:38.706320 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 16:29:38.706320 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 16:29:38.706320 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application result = self._query(query) May 19 16:29:38.706320 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 16:29:38.706320 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application conn.query(q) May 19 16:29:38.706320 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 16:29:38.706320 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 16:29:38.706320 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 16:29:38.707236 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application result.read() May 19 16:29:38.707236 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 16:29:38.707236 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 16:29:38.707236 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 16:29:38.707236 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 16:29:38.707236 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 16:29:38.707236 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 16:29:38.707236 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 16:29:38.707236 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 16:29:38.707236 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 16:29:38.707236 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:38.707236 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 16:29:38.707236 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:38.707236 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:38.707236 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 16:29:38.707236 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 16:29:38.707236 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 19 16:29:38.707236 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return project_ref.to_dict() May 19 16:29:38.708208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 16:29:38.708208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application next(self.gen) May 19 16:29:38.708208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 16:29:38.708208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application yield resource May 19 16:29:38.708208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 16:29:38.708208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application next(self.gen) May 19 16:29:38.708208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 16:29:38.708208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self.session.rollback() May 19 16:29:38.708208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 16:29:38.708208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self.force_reraise() May 19 16:29:38.708208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 16:29:38.708208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise self.value May 19 16:29:38.708208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 16:29:38.708208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 16:29:38.708208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 16:29:38.708208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application session.commit() May 19 16:29:38.708208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 16:29:38.709120 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 16:29:38.709120 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 16:29:38.709120 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._prepare_impl() May 19 16:29:38.709120 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 16:29:38.709120 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self.session.flush() May 19 16:29:38.709120 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 16:29:38.709120 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._flush(objects) May 19 16:29:38.709120 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 16:29:38.709120 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 16:29:38.709120 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 16:29:38.709120 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application compat.raise_( May 19 16:29:38.709120 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 16:29:38.709120 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception May 19 16:29:38.709120 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 16:29:38.709120 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application flush_context.execute() May 19 16:29:38.709120 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 16:29:38.709120 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rec.execute(self) May 19 16:29:38.710057 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 16:29:38.710057 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 16:29:38.710057 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 16:29:38.710057 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 16:29:38.710057 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 16:29:38.710057 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 16:29:38.710057 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 16:29:38.710057 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 16:29:38.710057 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 16:29:38.710057 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 16:29:38.710057 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 16:29:38.710057 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 16:29:38.710057 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 16:29:38.710057 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 16:29:38.710057 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 16:29:38.710853 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 16:29:38.710853 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 16:29:38.710853 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception May 19 16:29:38.710853 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 16:29:38.710853 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 16:29:38.710853 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 16:29:38.710853 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 16:29:38.710853 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 16:29:38.710853 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application result = self._query(query) May 19 16:29:38.710853 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 16:29:38.710853 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application conn.query(q) May 19 16:29:38.710853 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 16:29:38.710853 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 16:29:38.710853 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 16:29:38.710853 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application result.read() May 19 16:29:38.710853 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 16:29:38.710853 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 16:29:38.711715 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 16:29:38.711715 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 16:29:38.711715 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 16:29:38.711715 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 16:29:38.711715 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 16:29:38.711715 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 16:29:38.711715 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 16:29:38.711715 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 16:29:38.711715 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [parameters: {'id': 'e536d662b3fc43ab8b4e09f1779f9fa8', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 16:29:38.711715 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 16:29:38.711715 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:38.711715 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 16:29:38.711715 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:38.711715 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:38.711715 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 19 16:29:38.712499 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 19 16:29:38.712499 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 16:29:38.712499 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 16:29:38.712499 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 19 16:29:38.712499 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:38.712499 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 16:29:38.712499 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:38.712499 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:38.712499 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:38.712499 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:38.712499 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:38.712499 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:38.712499 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:38.712499 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:38.712499 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:38.712499 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:38.712499 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:38.716148 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:38.716148 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 19 16:29:38.716148 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 19 16:29:38.716148 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:38.716148 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:38.716148 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 19 16:29:38.716148 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.Conflict( May 19 16:29:38.716148 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e536d662b3fc43ab8b4e09f1779f9fa8. May 19 16:29:38.716148 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:38.716148 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 485/953] 10.222.0.36 () {60 vars in 1140 bytes} [Thu May 19 16:29:38 2022] POST /identity/v3/projects => generated 262 bytes in 206 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 19 16:29:38.716148 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa182b-7f11-4311-a170-24108b69e181 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:38.785781 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 469/954] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 16:29:38.819497 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abae3734-8804-47da-aa27-804bc918ae37 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:38.845581 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa182b-7f11-4311-a170-24108b69e181 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:38.883962 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc92554d-2c64-41e8-8595-551384e941cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:38.884771 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc92554d-2c64-41e8-8595-551384e941cb None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:38.885219 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc92554d-2c64-41e8-8595-551384e941cb None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:38.885463 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc92554d-2c64-41e8-8595-551384e941cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:38.900401 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc92554d-2c64-41e8-8595-551384e941cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:38.902380 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc92554d-2c64-41e8-8595-551384e941cb None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:38.919016 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-fc92554d-2c64-41e8-8595-551384e941cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:29:38.919016 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:38.919016 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:38.919016 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:38.919016 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:38.919016 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:38.919016 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:38.919016 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:38.919016 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:38.919016 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:38.919016 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:38.919016 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:38.919016 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 16:29:38.919016 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 16:29:38.919016 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 16:29:38.919986 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 16:29:38.919986 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:38.919986 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:38.919986 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:38.919986 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:38.919986 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:38.919986 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:38.919986 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:38.919986 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:38.919986 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:38.919986 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:38.919986 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:38.919986 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:38.919986 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:38.919986 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:38.919986 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:38.919986 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:38.920973 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 16:29:38.920973 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 16:29:38.920973 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 16:29:38.920973 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 16:29:38.920973 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 16:29:38.920973 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 16:29:38.920973 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:29:38.920973 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:38.924282 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 486/955] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:29:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 16:29:38.933342 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc92554d-2c64-41e8-8595-551384e941cb None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:38.938775 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abae3734-8804-47da-aa27-804bc918ae37 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:38.952888 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:38.955106 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:38.977449 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbed694-26d7-494d-8eec-1a615f416bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:38.978269 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbed694-26d7-494d-8eec-1a615f416bbf None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:38.978554 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbed694-26d7-494d-8eec-1a615f416bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:38.978832 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbed694-26d7-494d-8eec-1a615f416bbf None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:38.979496 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbed694-26d7-494d-8eec-1a615f416bbf None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:38.980880 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbed694-26d7-494d-8eec-1a615f416bbf None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:38.993451 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-5fbed694-26d7-494d-8eec-1a615f416bbf None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 16:29:38.993451 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:38.993451 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 16:29:38.993451 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 16:29:38.993451 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 16:29:38.993451 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 16:29:38.993451 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 16:29:38.993451 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:29:38.993451 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 16:29:38.993451 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:38.993451 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 16:29:38.993451 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:38.993451 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:38.993451 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:38.993451 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:38.993451 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:38.993451 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:38.994581 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:38.994581 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:38.994581 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:38.994581 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:38.994581 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:38.994581 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:38.994581 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 16:29:38.994581 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 16:29:38.994581 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 16:29:38.994581 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 16:29:38.994581 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:38.994581 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:38.994581 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:38.994581 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:38.994581 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:38.994581 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:38.995510 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:38.995510 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:38.995510 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:38.995510 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:38.995510 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:38.995510 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:38.995510 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:38.995510 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:38.995510 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:38.995510 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:38.995510 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 16:29:38.995510 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:29:38.995510 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 16:29:38.995510 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:38.999928 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 470/956] 10.222.0.36 () {56 vars in 1126 bytes} [Thu May 19 16:29:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 16:29:39.000831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbed694-26d7-494d-8eec-1a615f416bbf None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:39.054679 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc92554d-2c64-41e8-8595-551384e941cb None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:39.094075 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37052daa-398d-4b26-94a4-95d5a49cf5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:39.095169 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37052daa-398d-4b26-94a4-95d5a49cf5ae None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:39.095608 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37052daa-398d-4b26-94a4-95d5a49cf5ae None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:39.096141 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37052daa-398d-4b26-94a4-95d5a49cf5ae None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:39.097229 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37052daa-398d-4b26-94a4-95d5a49cf5ae None admin] RBAC: Adding query filter params (name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:39.097844 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37052daa-398d-4b26-94a4-95d5a49cf5ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:39.099482 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37052daa-398d-4b26-94a4-95d5a49cf5ae None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:39.103481 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbed694-26d7-494d-8eec-1a615f416bbf None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:39.128119 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 487/957] 10.222.0.36 () {56 vars in 1112 bytes} [Thu May 19 16:29:38 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:29:39.143137 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd94759-7160-47d9-bc1f-fa7e28abc7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:39.143918 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd94759-7160-47d9-bc1f-fa7e28abc7c6 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:39.144472 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd94759-7160-47d9-bc1f-fa7e28abc7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:39.144797 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd94759-7160-47d9-bc1f-fa7e28abc7c6 None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:39.145711 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd94759-7160-47d9-bc1f-fa7e28abc7c6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:39.146027 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd94759-7160-47d9-bc1f-fa7e28abc7c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:39.147941 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd94759-7160-47d9-bc1f-fa7e28abc7c6 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:39.154107 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 471/958] 10.222.0.36 () {56 vars in 1122 bytes} [Thu May 19 16:29:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:39.165280 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37052daa-398d-4b26-94a4-95d5a49cf5ae None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:39.261179 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:39.286946 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37052daa-398d-4b26-94a4-95d5a49cf5ae None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:39.325590 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e57f0-8ba2-499f-badb-f00940a41ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:39.326190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e57f0-8ba2-499f-badb-f00940a41ae9 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:39.326480 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e57f0-8ba2-499f-badb-f00940a41ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:39.326762 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e57f0-8ba2-499f-badb-f00940a41ae9 None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:39.327529 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9e57f0-8ba2-499f-badb-f00940a41ae9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:39.327846 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9e57f0-8ba2-499f-badb-f00940a41ae9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:39.329268 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9e57f0-8ba2-499f-badb-f00940a41ae9 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:39.335076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 488/959] 10.222.0.36 () {56 vars in 1122 bytes} [Thu May 19 16:29:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:39.340592 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd94759-7160-47d9-bc1f-fa7e28abc7c6 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:39.387749 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:39.450524 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd94759-7160-47d9-bc1f-fa7e28abc7c6 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:39.492287 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc34917c-e1f9-4e90-ba4c-de97861309d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:39.492976 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc34917c-e1f9-4e90-ba4c-de97861309d9 None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:39.493310 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc34917c-e1f9-4e90-ba4c-de97861309d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:39.493597 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc34917c-e1f9-4e90-ba4c-de97861309d9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:39.494392 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc34917c-e1f9-4e90-ba4c-de97861309d9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:39.495986 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc34917c-e1f9-4e90-ba4c-de97861309d9 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:39.517950 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 472/960] 10.222.0.36 () {60 vars in 1144 bytes} [Thu May 19 16:29:39 2022] POST /identity/v3/endpoints => generated 349 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:29:39.816429 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c24a55-56e2-4f7f-aa1e-bfef414f8086 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:39.816813 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c24a55-56e2-4f7f-aa1e-bfef414f8086 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:39.817075 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c24a55-56e2-4f7f-aa1e-bfef414f8086 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:39.818797 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 489/961] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:29:39.824473 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361fbd8d-e6d7-49a0-9a85-d78e6470eb19 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:39.824793 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361fbd8d-e6d7-49a0-9a85-d78e6470eb19 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:39.825061 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361fbd8d-e6d7-49a0-9a85-d78e6470eb19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:39.844497 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-361fbd8d-e6d7-49a0-9a85-d78e6470eb19 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:39.845757 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-361fbd8d-e6d7-49a0-9a85-d78e6470eb19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:39.933294 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 473/962] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:29:39.950735 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c24a55-56e2-4f7f-aa1e-bfef414f8086 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:40.058420 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c24a55-56e2-4f7f-aa1e-bfef414f8086 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:40.094940 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c735bc03-28e6-44d7-ae37-640ad2da64dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:40.095691 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735bc03-28e6-44d7-ae37-640ad2da64dd None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:40.095983 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735bc03-28e6-44d7-ae37-640ad2da64dd None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:40.096267 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735bc03-28e6-44d7-ae37-640ad2da64dd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:40.106959 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c735bc03-28e6-44d7-ae37-640ad2da64dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:29:40.116000 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-c735bc03-28e6-44d7-ae37-640ad2da64dd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 16:29:40.116000 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:40.116000 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:40.116000 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:40.116000 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:40.116000 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:40.116000 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:40.116000 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:40.116000 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:40.116000 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:40.116000 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:40.116000 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:40.116000 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:29:40.116000 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:29:40.116000 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:29:40.116000 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:29:40.116957 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:29:40.116957 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:29:40.116957 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:29:40.116957 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:29:40.116957 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:40.116957 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:40.116957 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:40.116957 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:40.116957 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:40.116957 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:40.116957 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:40.116957 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:40.116957 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:40.116957 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:40.116957 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:40.116957 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:40.118513 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:40.118513 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:40.118513 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:40.118513 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:40.118513 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:29:40.118513 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:29:40.118513 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:29:40.118513 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:29:40.118513 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:29:40.118513 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:29:40.118513 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 16:29:40.118513 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:40.118513 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 490/963] 10.222.0.36 () {56 vars in 1120 bytes} [Thu May 19 16:29:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:40.123257 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361fbd8d-e6d7-49a0-9a85-d78e6470eb19 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:40.153584 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:40.229746 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361fbd8d-e6d7-49a0-9a85-d78e6470eb19 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:40.265181 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e445b73-fba4-4148-9c92-36e6dc65440c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:40.265920 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e445b73-fba4-4148-9c92-36e6dc65440c None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:40.266211 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e445b73-fba4-4148-9c92-36e6dc65440c None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:40.266491 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e445b73-fba4-4148-9c92-36e6dc65440c None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:40.267254 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e445b73-fba4-4148-9c92-36e6dc65440c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:40.267563 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e445b73-fba4-4148-9c92-36e6dc65440c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:40.269291 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e445b73-fba4-4148-9c92-36e6dc65440c None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:40.282742 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 474/964] 10.222.0.36 () {56 vars in 1115 bytes} [Thu May 19 16:29:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:40.284117 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c735bc03-28e6-44d7-ae37-640ad2da64dd None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:40.302168 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:40.390426 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c735bc03-28e6-44d7-ae37-640ad2da64dd None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:40.432068 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6398582f-7257-4103-8120-61d2c0e4a598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:40.432887 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398582f-7257-4103-8120-61d2c0e4a598 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:40.433194 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398582f-7257-4103-8120-61d2c0e4a598 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:40.433469 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398582f-7257-4103-8120-61d2c0e4a598 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:40.445030 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6398582f-7257-4103-8120-61d2c0e4a598 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:40.446536 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6398582f-7257-4103-8120-61d2c0e4a598 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:40.464344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-6398582f-7257-4103-8120-61d2c0e4a598 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:40.464344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:40.464344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:40.464344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:40.464344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:40.464344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:40.464344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:40.464344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:40.464344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:40.464344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:40.464344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:40.464344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:40.464344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:29:40.464344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:29:40.464344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:29:40.466457 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:29:40.466457 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:40.466457 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:40.466457 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:40.466457 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:40.466457 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:40.466457 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:40.466457 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:40.466457 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:40.466457 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:40.466457 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:40.466457 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:40.466457 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:40.466457 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:40.466457 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:40.466457 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:40.466457 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:40.469122 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:29:40.469122 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:29:40.469122 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:40.469122 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:40.469122 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 491/965] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:29:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:40.471955 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e445b73-fba4-4148-9c92-36e6dc65440c None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:40.562120 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:40.578840 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e445b73-fba4-4148-9c92-36e6dc65440c None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:40.614355 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f3639-7043-46c4-aed9-9dcd9ebdf408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:40.615082 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f3639-7043-46c4-aed9-9dcd9ebdf408 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:40.615508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f3639-7043-46c4-aed9-9dcd9ebdf408 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:40.615791 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f3639-7043-46c4-aed9-9dcd9ebdf408 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:40.616645 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f3639-7043-46c4-aed9-9dcd9ebdf408 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:40.616958 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f3639-7043-46c4-aed9-9dcd9ebdf408 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:40.618408 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f3639-7043-46c4-aed9-9dcd9ebdf408 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:40.635392 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 475/966] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:29:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:40.636786 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6398582f-7257-4103-8120-61d2c0e4a598 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:40.694790 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:40.744362 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6398582f-7257-4103-8120-61d2c0e4a598 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:40.780495 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6806338-c3c6-4d29-8084-b73c3576ea1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:40.781365 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6806338-c3c6-4d29-8084-b73c3576ea1a None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:40.781659 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6806338-c3c6-4d29-8084-b73c3576ea1a None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:40.781941 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6806338-c3c6-4d29-8084-b73c3576ea1a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:40.789979 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6806338-c3c6-4d29-8084-b73c3576ea1a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:40.791428 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6806338-c3c6-4d29-8084-b73c3576ea1a None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:40.806763 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-f6806338-c3c6-4d29-8084-b73c3576ea1a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 16:29:40.806763 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:40.806763 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:40.806763 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:40.806763 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:40.806763 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:40.806763 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:40.806763 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:40.806763 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:40.806763 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:40.806763 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:40.806763 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:40.806763 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 16:29:40.806763 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 16:29:40.806763 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 16:29:40.806763 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 16:29:40.808877 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:40.808877 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:40.808877 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 16:29:40.808877 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:29:40.808877 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 16:29:40.808877 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:29:40.808877 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:40.808877 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:40.808877 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:40.808877 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:40.808877 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:40.808877 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:40.808877 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:40.808877 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:40.808877 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:40.808877 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:40.808877 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:40.808877 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:40.811052 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:40.811052 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:40.811052 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 16:29:40.811052 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 16:29:40.811052 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 16:29:40.811052 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 16:29:40.811052 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 16:29:40.811052 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 16:29:40.811052 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 16:29:40.811052 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:40.811052 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 492/967] 10.222.0.36 () {56 vars in 1117 bytes} [Thu May 19 16:29:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:40.817197 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f3639-7043-46c4-aed9-9dcd9ebdf408 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:40.935857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f3639-7043-46c4-aed9-9dcd9ebdf408 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:40.972225 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e915f28-e9ac-4703-b1b4-79abcac99622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:40.972822 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e915f28-e9ac-4703-b1b4-79abcac99622 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:40.973114 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e915f28-e9ac-4703-b1b4-79abcac99622 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:40.973428 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e915f28-e9ac-4703-b1b4-79abcac99622 None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:40.974283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e915f28-e9ac-4703-b1b4-79abcac99622 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:40.974600 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e915f28-e9ac-4703-b1b4-79abcac99622 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:40.976054 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e915f28-e9ac-4703-b1b4-79abcac99622 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:40.998426 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 476/968] 10.222.0.36 () {56 vars in 1149 bytes} [Thu May 19 16:29:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:41.000296 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6806338-c3c6-4d29-8084-b73c3576ea1a None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:41.081143 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d61321-4c59-400e-b611-1b5367062208 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:41.081486 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d61321-4c59-400e-b611-1b5367062208 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:41.081754 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d61321-4c59-400e-b611-1b5367062208 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:41.083298 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 477/969] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 16:29:41.089432 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d44d56e-96c6-49ca-a3fe-83ee17a4723a None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:41.089975 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d44d56e-96c6-49ca-a3fe-83ee17a4723a None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:41.090476 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d44d56e-96c6-49ca-a3fe-83ee17a4723a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:41.097591 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:41.108993 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-1d44d56e-96c6-49ca-a3fe-83ee17a4723a None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:41.110617 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-1d44d56e-96c6-49ca-a3fe-83ee17a4723a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:41.112008 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6806338-c3c6-4d29-8084-b73c3576ea1a None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:41.127043 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:41.149466 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bda768-0073-4722-91c3-2aa5266df668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:41.150177 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bda768-0073-4722-91c3-2aa5266df668 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:41.150490 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bda768-0073-4722-91c3-2aa5266df668 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:41.150852 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bda768-0073-4722-91c3-2aa5266df668 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:41.189911 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 478/970] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 16:29:41.214680 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d44d56e-96c6-49ca-a3fe-83ee17a4723a None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:41.231100 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bda768-0073-4722-91c3-2aa5266df668 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:41.232916 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bda768-0073-4722-91c3-2aa5266df668 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:41.345520 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d44d56e-96c6-49ca-a3fe-83ee17a4723a None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:41.346157 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 493/971] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:29:40 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 16:29:41.347673 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bda768-0073-4722-91c3-2aa5266df668 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:41.363845 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:41.385572 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:41.396785 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb75fda-8aba-4b2b-9d60-05734b2b50ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:41.397625 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb75fda-8aba-4b2b-9d60-05734b2b50ab None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:41.397987 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb75fda-8aba-4b2b-9d60-05734b2b50ab None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:41.398341 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb75fda-8aba-4b2b-9d60-05734b2b50ab None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:41.399778 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb75fda-8aba-4b2b-9d60-05734b2b50ab None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:41.401779 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb75fda-8aba-4b2b-9d60-05734b2b50ab None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:41.425375 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 479/972] 10.222.0.36 () {60 vars in 1141 bytes} [Thu May 19 16:29:41 2022] POST /identity/v3/services => generated 242 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:29:41.475298 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bda768-0073-4722-91c3-2aa5266df668 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:41.527847 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeef0c2-2a08-4117-b50a-8aadb747b20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:41.529003 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeef0c2-2a08-4117-b50a-8aadb747b20f None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:41.529487 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeef0c2-2a08-4117-b50a-8aadb747b20f None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:41.529941 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeef0c2-2a08-4117-b50a-8aadb747b20f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:41.549502 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeef0c2-2a08-4117-b50a-8aadb747b20f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:41.551492 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeef0c2-2a08-4117-b50a-8aadb747b20f None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:41.572058 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-4aeef0c2-2a08-4117-b50a-8aadb747b20f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:41.572058 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:41.572058 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:41.572058 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:41.572058 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:41.572058 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:41.572058 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:41.572058 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:41.572058 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:41.572058 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:41.572058 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:41.572058 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:41.572058 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:29:41.572058 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:29:41.572058 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:29:41.573078 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:29:41.573078 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:41.573078 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:41.573078 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:41.573078 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:41.573078 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:41.573078 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:41.573078 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:41.573078 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:41.573078 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:41.573078 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:41.573078 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:41.573078 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:41.573078 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:41.573078 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:41.573078 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:41.573078 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:41.574463 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:29:41.574463 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:29:41.574463 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:41.574463 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:41.574463 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 494/973] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:29:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:41.580921 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb75fda-8aba-4b2b-9d60-05734b2b50ab None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:41.584026 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcbb78e-61fd-44fb-9f53-ece9f8447afd None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:41.584323 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcbb78e-61fd-44fb-9f53-ece9f8447afd None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:41.584601 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcbb78e-61fd-44fb-9f53-ece9f8447afd None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:41.586335 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 495/974] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:29:41.591892 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f60c657-cfe8-4ddf-ad50-9584bcd8af90 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:41.592197 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f60c657-cfe8-4ddf-ad50-9584bcd8af90 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:41.592464 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f60c657-cfe8-4ddf-ad50-9584bcd8af90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:41.612044 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-2f60c657-cfe8-4ddf-ad50-9584bcd8af90 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:41.613672 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-2f60c657-cfe8-4ddf-ad50-9584bcd8af90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:41.687931 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:41.704083 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 496/975] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:41 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 16:29:41.711136 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb75fda-8aba-4b2b-9d60-05734b2b50ab None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:41.731574 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f60c657-cfe8-4ddf-ad50-9584bcd8af90 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:41.766309 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4318e80-8256-49e1-859a-0e8039e34764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:41.767008 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4318e80-8256-49e1-859a-0e8039e34764 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:41.767300 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4318e80-8256-49e1-859a-0e8039e34764 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:41.767586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4318e80-8256-49e1-859a-0e8039e34764 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:41.768609 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4318e80-8256-49e1-859a-0e8039e34764 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:41.769204 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4318e80-8256-49e1-859a-0e8039e34764 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:41.770607 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4318e80-8256-49e1-859a-0e8039e34764 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:41.791329 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 480/976] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:29:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:41.793510 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4318e80-8256-49e1-859a-0e8039e34764 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:41.860297 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f60c657-cfe8-4ddf-ad50-9584bcd8af90 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:41.866136 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:41.907253 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ff3426-d8de-4cfa-bf06-f82f9a9ffdf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:41.908050 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff3426-d8de-4cfa-bf06-f82f9a9ffdf3 None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:41.908337 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff3426-d8de-4cfa-bf06-f82f9a9ffdf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:41.908640 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff3426-d8de-4cfa-bf06-f82f9a9ffdf3 None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:41.909448 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ff3426-d8de-4cfa-bf06-f82f9a9ffdf3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:41.910910 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ff3426-d8de-4cfa-bf06-f82f9a9ffdf3 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:41.927593 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4318e80-8256-49e1-859a-0e8039e34764 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:41.928523 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.sql.core [None req-75ff3426-d8de-4cfa-bf06-f82f9a9ffdf3 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 16:29:41.928523 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 16:29:41.928523 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [parameters: {'user_id': '6ca0bc527d22467dbdf207af60ef83b3', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 16:29:41.928523 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80466) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 16:29:41.936729 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-75ff3426-d8de-4cfa-bf06-f82f9a9ffdf3 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 16:29:41.936729 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:41.936729 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 16:29:41.936729 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 16:29:41.936729 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 16:29:41.936729 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 16:29:41.936729 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 16:29:41.936729 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application result = self._query(query) May 19 16:29:41.936729 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 16:29:41.936729 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application conn.query(q) May 19 16:29:41.936729 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 16:29:41.936729 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 16:29:41.936729 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 16:29:41.936729 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application result.read() May 19 16:29:41.936729 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 16:29:41.937653 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 16:29:41.937653 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 16:29:41.937653 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 16:29:41.937653 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 16:29:41.937653 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 16:29:41.937653 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 16:29:41.937653 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 16:29:41.937653 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 16:29:41.937653 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:41.937653 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 16:29:41.937653 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:41.937653 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:41.937653 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 16:29:41.937653 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 16:29:41.937653 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 19 16:29:41.937653 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 19 16:29:41.937653 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 16:29:41.937653 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application next(self.gen) May 19 16:29:41.938638 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 16:29:41.938638 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application yield resource May 19 16:29:41.938638 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 16:29:41.938638 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application next(self.gen) May 19 16:29:41.938638 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 16:29:41.938638 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self.session.rollback() May 19 16:29:41.938638 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 16:29:41.938638 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self.force_reraise() May 19 16:29:41.938638 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 16:29:41.938638 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise self.value May 19 16:29:41.938638 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 16:29:41.938638 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 16:29:41.938638 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 16:29:41.938638 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application session.commit() May 19 16:29:41.938638 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 16:29:41.938638 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 16:29:41.938638 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 16:29:41.939529 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._prepare_impl() May 19 16:29:41.939529 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 16:29:41.939529 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self.session.flush() May 19 16:29:41.939529 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 16:29:41.939529 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._flush(objects) May 19 16:29:41.939529 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 16:29:41.939529 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 16:29:41.939529 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 16:29:41.939529 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application compat.raise_( May 19 16:29:41.939529 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 16:29:41.939529 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception May 19 16:29:41.939529 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 16:29:41.939529 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application flush_context.execute() May 19 16:29:41.939529 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 16:29:41.939529 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rec.execute(self) May 19 16:29:41.939529 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 16:29:41.939529 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 16:29:41.940501 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 16:29:41.940501 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 16:29:41.940501 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 19 16:29:41.940501 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application result = connection._execute_20( May 19 16:29:41.940501 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 16:29:41.940501 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 16:29:41.940501 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 16:29:41.940501 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 16:29:41.940501 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 16:29:41.940501 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 16:29:41.940501 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 16:29:41.940501 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 16:29:41.940501 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 16:29:41.940501 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 16:29:41.940501 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 16:29:41.940501 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception May 19 16:29:41.941368 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 16:29:41.941368 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 16:29:41.941368 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 16:29:41.941368 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 16:29:41.941368 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 16:29:41.941368 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application result = self._query(query) May 19 16:29:41.941368 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 16:29:41.941368 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application conn.query(q) May 19 16:29:41.941368 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 16:29:41.941368 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 16:29:41.941368 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 16:29:41.941368 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application result.read() May 19 16:29:41.941368 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 16:29:41.941368 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 16:29:41.941368 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 16:29:41.941368 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 16:29:41.942190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 16:29:41.942190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 16:29:41.942190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 16:29:41.942190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 16:29:41.942190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 16:29:41.942190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 16:29:41.942190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [parameters: {'user_id': '6ca0bc527d22467dbdf207af60ef83b3', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 16:29:41.942190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 16:29:41.942190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:41.942190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 16:29:41.942190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:41.942190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:41.942190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:41.942190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:41.942190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:41.943003 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:41.943003 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:41.943003 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:41.943003 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:41.943003 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:41.943003 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:41.943003 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:41.943003 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 19 16:29:41.943003 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 19 16:29:41.943003 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:41.943003 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:41.943003 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 16:29:41.943003 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:29:41.943003 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 16:29:41.943003 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:29:41.943003 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 19 16:29:41.943003 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 19 16:29:41.951721 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 19 16:29:41.951721 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 19 16:29:41.951721 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 16:29:41.951721 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 16:29:41.951721 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 16:29:41.951721 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:41.951721 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 497/977] 10.222.0.36 () {60 vars in 1131 bytes} [Thu May 19 16:29:41 2022] POST /identity/v3/users => generated 160 bytes in 207 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 19 16:29:41.951721 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1eb217-c967-4dce-888e-0fe55be24b0d None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:41.951721 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1eb217-c967-4dce-888e-0fe55be24b0d None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:41.951721 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1eb217-c967-4dce-888e-0fe55be24b0d None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:41.954890 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 498/978] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:29:41.954890 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1eb217-c967-4dce-888e-0fe55be24b0d None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:41.976198 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189809bb-dc7b-4ec5-8f6c-ad79a6b5c216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:41.977224 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189809bb-dc7b-4ec5-8f6c-ad79a6b5c216 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:41.977496 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189809bb-dc7b-4ec5-8f6c-ad79a6b5c216 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:41.977841 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189809bb-dc7b-4ec5-8f6c-ad79a6b5c216 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:41.984915 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189809bb-dc7b-4ec5-8f6c-ad79a6b5c216 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:41.986785 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189809bb-dc7b-4ec5-8f6c-ad79a6b5c216 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:42.000831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-189809bb-dc7b-4ec5-8f6c-ad79a6b5c216 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:29:42.000831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:42.000831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:42.000831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:42.000831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:42.000831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:42.000831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:42.000831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:42.000831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:42.000831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:42.000831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:42.000831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:42.000831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 16:29:42.000831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 16:29:42.000831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 16:29:42.001891 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 16:29:42.001891 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:42.001891 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:42.001891 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:42.001891 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:42.001891 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:42.001891 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:42.001891 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:42.001891 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:42.001891 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:42.001891 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:42.001891 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:42.001891 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:42.001891 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:42.001891 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:42.001891 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:42.001891 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:42.002815 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 16:29:42.002815 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 16:29:42.002815 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 16:29:42.002815 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 16:29:42.002815 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 16:29:42.002815 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 16:29:42.002815 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:29:42.002815 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:42.002815 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 481/979] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:29:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:42.003662 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2829e3-03e5-4cc8-baf7-300ff78d533f None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:42.003954 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2829e3-03e5-4cc8-baf7-300ff78d533f None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:42.004233 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2829e3-03e5-4cc8-baf7-300ff78d533f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:42.017678 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:42.023498 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-6f2829e3-03e5-4cc8-baf7-300ff78d533f None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:42.024698 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-6f2829e3-03e5-4cc8-baf7-300ff78d533f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:42.072505 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1eb217-c967-4dce-888e-0fe55be24b0d None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:42.097615 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 482/980] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:42 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:29:42.101402 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2829e3-03e5-4cc8-baf7-300ff78d533f None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:42.120772 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f24ee67-979b-441b-bf12-a775a59929da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:42.122117 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f24ee67-979b-441b-bf12-a775a59929da None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:42.122580 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f24ee67-979b-441b-bf12-a775a59929da None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:42.123001 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f24ee67-979b-441b-bf12-a775a59929da None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:42.131883 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f24ee67-979b-441b-bf12-a775a59929da None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:42.133777 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f24ee67-979b-441b-bf12-a775a59929da None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:42.149403 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-0f24ee67-979b-441b-bf12-a775a59929da None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 16:29:42.149403 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:42.149403 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:42.149403 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:42.149403 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:42.149403 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:42.149403 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:42.149403 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:42.149403 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:42.149403 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:42.149403 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:42.149403 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:42.149403 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 16:29:42.149403 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 16:29:42.149403 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 16:29:42.149403 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 16:29:42.150419 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:42.150419 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:42.150419 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 16:29:42.150419 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:29:42.150419 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 16:29:42.150419 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:29:42.150419 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:42.150419 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:42.150419 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:42.150419 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:42.150419 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:42.150419 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:42.150419 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:42.150419 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:42.150419 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:42.150419 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:42.150419 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:42.150419 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:42.151377 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:42.151377 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:42.151377 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 16:29:42.151377 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 16:29:42.151377 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 16:29:42.151377 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 16:29:42.151377 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 16:29:42.151377 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 16:29:42.151377 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 16:29:42.151377 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:42.151377 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 499/981] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:29:41 2022] GET /identity/v3/users/cinder => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:42.151377 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f24ee67-979b-441b-bf12-a775a59929da None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:42.161729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:42.226876 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2829e3-03e5-4cc8-baf7-300ff78d533f None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:42.262975 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f24ee67-979b-441b-bf12-a775a59929da None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:42.266574 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526331cc-9c68-4128-b9dd-e4fb6a9073f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:42.267249 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526331cc-9c68-4128-b9dd-e4fb6a9073f1 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:42.267535 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526331cc-9c68-4128-b9dd-e4fb6a9073f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:42.267816 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526331cc-9c68-4128-b9dd-e4fb6a9073f1 None admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:42.268628 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526331cc-9c68-4128-b9dd-e4fb6a9073f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:42.269238 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526331cc-9c68-4128-b9dd-e4fb6a9073f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:42.270697 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526331cc-9c68-4128-b9dd-e4fb6a9073f1 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:42.278856 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:42.287637 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 483/982] 10.222.0.36 () {56 vars in 1148 bytes} [Thu May 19 16:29:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:42.287637 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526331cc-9c68-4128-b9dd-e4fb6a9073f1 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:42.306688 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7059cc-d9c0-4958-8c8f-ad91fd42a04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:42.307610 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7059cc-d9c0-4958-8c8f-ad91fd42a04c None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:42.307900 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7059cc-d9c0-4958-8c8f-ad91fd42a04c None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:42.308618 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7059cc-d9c0-4958-8c8f-ad91fd42a04c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:42.309323 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7059cc-d9c0-4958-8c8f-ad91fd42a04c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:42.310785 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7059cc-d9c0-4958-8c8f-ad91fd42a04c None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:42.324035 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-9a7059cc-d9c0-4958-8c8f-ad91fd42a04c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 16:29:42.324035 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:42.324035 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 16:29:42.324035 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 16:29:42.324035 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 16:29:42.324035 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 16:29:42.324035 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 16:29:42.324035 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:29:42.324035 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 16:29:42.324035 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:42.324035 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 16:29:42.324035 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:42.324035 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:42.324035 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:42.324035 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:42.324035 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:42.324035 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:42.325018 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:42.325018 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:42.325018 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:42.325018 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:42.325018 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:42.325018 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:42.325018 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 16:29:42.325018 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 16:29:42.325018 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 16:29:42.325018 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 16:29:42.325018 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:42.325018 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:42.325018 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:42.325018 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:42.325018 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:42.325018 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:42.331727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:42.331727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:42.331727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:42.331727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:42.331727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:42.331727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:42.331727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:42.331727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:42.331727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:42.331727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:42.331727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 16:29:42.331727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:29:42.331727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 16:29:42.331727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:42.331727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 500/983] 10.222.0.36 () {56 vars in 1126 bytes} [Thu May 19 16:29:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:42.332534 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7059cc-d9c0-4958-8c8f-ad91fd42a04c None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:42.398997 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526331cc-9c68-4128-b9dd-e4fb6a9073f1 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:42.400655 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:42.435227 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16098948-e58d-43ba-bee1-db31af1202cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:42.436830 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16098948-e58d-43ba-bee1-db31af1202cd None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:42.437420 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16098948-e58d-43ba-bee1-db31af1202cd None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:42.437704 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16098948-e58d-43ba-bee1-db31af1202cd None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:42.438519 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16098948-e58d-43ba-bee1-db31af1202cd None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:42.438832 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16098948-e58d-43ba-bee1-db31af1202cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:42.439721 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7059cc-d9c0-4958-8c8f-ad91fd42a04c None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:42.440276 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16098948-e58d-43ba-bee1-db31af1202cd None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:42.476788 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd17c4e-e4dc-47dd-9f56-d5d8ee065e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:42.488745 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd17c4e-e4dc-47dd-9f56-d5d8ee065e4f None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:42.488947 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 484/984] 10.222.0.36 () {56 vars in 1101 bytes} [Thu May 19 16:29:42 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:42.489031 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd17c4e-e4dc-47dd-9f56-d5d8ee065e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:42.489031 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd17c4e-e4dc-47dd-9f56-d5d8ee065e4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:42.489216 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16098948-e58d-43ba-bee1-db31af1202cd None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:42.489291 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd17c4e-e4dc-47dd-9f56-d5d8ee065e4f None admin] RBAC: Adding query filter params (role.id=58a72d6ac3ff4e278e79059c4fe2ec5a, scope.project.id=631c76794a124d6da22c626e590e4352, user.id=a30dc1dd2b00463bb3c7957864c56043) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:42.489291 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd17c4e-e4dc-47dd-9f56-d5d8ee065e4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:42.489291 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd17c4e-e4dc-47dd-9f56-d5d8ee065e4f None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:42.489291 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 501/985] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:29:42 2022] GET /identity/v3/role_assignments?user.id=a30dc1dd2b00463bb3c7957864c56043&scope.project.id=631c76794a124d6da22c626e590e4352&role.id=58a72d6ac3ff4e278e79059c4fe2ec5a => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:42.624350 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16098948-e58d-43ba-bee1-db31af1202cd None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:42.665508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744aa2ca-67b9-4eb4-bfc7-259264d2fff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:42.666521 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744aa2ca-67b9-4eb4-bfc7-259264d2fff4 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:42.667024 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744aa2ca-67b9-4eb4-bfc7-259264d2fff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:42.667517 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744aa2ca-67b9-4eb4-bfc7-259264d2fff4 None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:42.668491 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744aa2ca-67b9-4eb4-bfc7-259264d2fff4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:42.669023 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744aa2ca-67b9-4eb4-bfc7-259264d2fff4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:42.671317 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744aa2ca-67b9-4eb4-bfc7-259264d2fff4 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:42.678005 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 485/986] 10.222.0.36 () {56 vars in 1122 bytes} [Thu May 19 16:29:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:42.689195 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd17c4e-e4dc-47dd-9f56-d5d8ee065e4f None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:42.734370 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:42.794658 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd17c4e-e4dc-47dd-9f56-d5d8ee065e4f None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:42.830676 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73046afc-b56e-49c3-b2ad-e87ed7fbe17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:42.831375 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73046afc-b56e-49c3-b2ad-e87ed7fbe17c None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:42.831658 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73046afc-b56e-49c3-b2ad-e87ed7fbe17c None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:42.831936 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73046afc-b56e-49c3-b2ad-e87ed7fbe17c None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:42.832654 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73046afc-b56e-49c3-b2ad-e87ed7fbe17c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:42.832971 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73046afc-b56e-49c3-b2ad-e87ed7fbe17c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:42.834380 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73046afc-b56e-49c3-b2ad-e87ed7fbe17c None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:42.839776 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 502/987] 10.222.0.36 () {56 vars in 1122 bytes} [Thu May 19 16:29:42 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:42.846267 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744aa2ca-67b9-4eb4-bfc7-259264d2fff4 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:42.881671 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:42.974957 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744aa2ca-67b9-4eb4-bfc7-259264d2fff4 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:43.011003 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beeecbd-a67a-4aea-b2cf-a85c7709b9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:43.011658 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beeecbd-a67a-4aea-b2cf-a85c7709b9e9 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:43.011940 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beeecbd-a67a-4aea-b2cf-a85c7709b9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:43.012214 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beeecbd-a67a-4aea-b2cf-a85c7709b9e9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:43.013017 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beeecbd-a67a-4aea-b2cf-a85c7709b9e9 None admin] RBAC: Adding query filter params (interface=admin, service_id=617cce2fd363454a97312ee4379d57c4, region_id=RegionOne) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:43.013352 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beeecbd-a67a-4aea-b2cf-a85c7709b9e9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:43.014758 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beeecbd-a67a-4aea-b2cf-a85c7709b9e9 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:43.020282 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 486/988] 10.222.0.36 () {56 vars in 1249 bytes} [Thu May 19 16:29:42 2022] GET /identity/v3/endpoints?service_id=617cce2fd363454a97312ee4379d57c4&interface=admin®ion_id=RegionOne => generated 193 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:43.025518 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73046afc-b56e-49c3-b2ad-e87ed7fbe17c None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:43.130886 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73046afc-b56e-49c3-b2ad-e87ed7fbe17c None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:43.138607 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:43.167730 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf62973d-dfd5-45ed-a35a-b60cb9fcaa03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:43.168403 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf62973d-dfd5-45ed-a35a-b60cb9fcaa03 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:43.168695 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf62973d-dfd5-45ed-a35a-b60cb9fcaa03 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:43.168978 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf62973d-dfd5-45ed-a35a-b60cb9fcaa03 None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:43.169706 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf62973d-dfd5-45ed-a35a-b60cb9fcaa03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:43.171733 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf62973d-dfd5-45ed-a35a-b60cb9fcaa03 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:43.185417 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 503/989] 10.222.0.36 () {56 vars in 1087 bytes} [Thu May 19 16:29:43 2022] GET /identity/v3/services => generated 1727 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:29:43.800309 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f0e666-efc9-4098-b790-395010694ec3 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:43.800826 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f0e666-efc9-4098-b790-395010694ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:43.801049 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f0e666-efc9-4098-b790-395010694ec3 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:43.802499 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 487/990] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:29:43.807528 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fff5e1-ecdf-4b45-ada0-c8a7ac377e13 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:43.807829 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fff5e1-ecdf-4b45-ada0-c8a7ac377e13 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:43.808113 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fff5e1-ecdf-4b45-ada0-c8a7ac377e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:43.825819 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-23fff5e1-ecdf-4b45-ada0-c8a7ac377e13 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:43.827897 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-23fff5e1-ecdf-4b45-ada0-c8a7ac377e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:43.902568 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 504/991] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:43 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:29:43.925188 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f0e666-efc9-4098-b790-395010694ec3 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:44.017816 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:44.032141 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f0e666-efc9-4098-b790-395010694ec3 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:44.067909 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690940e9-f362-470a-a812-0cfb1ed1b591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:44.068626 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690940e9-f362-470a-a812-0cfb1ed1b591 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:44.068915 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690940e9-f362-470a-a812-0cfb1ed1b591 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:44.069216 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690940e9-f362-470a-a812-0cfb1ed1b591 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:44.069879 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690940e9-f362-470a-a812-0cfb1ed1b591 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:44.071249 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690940e9-f362-470a-a812-0cfb1ed1b591 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:44.083497 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-690940e9-f362-470a-a812-0cfb1ed1b591 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 16:29:44.083497 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:44.083497 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 16:29:44.083497 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 16:29:44.083497 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 16:29:44.083497 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 16:29:44.083497 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 16:29:44.083497 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:29:44.083497 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 16:29:44.083497 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:44.083497 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 16:29:44.083497 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:44.083497 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:44.083497 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:44.083497 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:44.083497 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:44.083497 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:44.084698 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:44.084698 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:44.084698 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:44.084698 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:44.084698 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:44.084698 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:44.084698 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 16:29:44.084698 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 16:29:44.084698 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 16:29:44.084698 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 16:29:44.084698 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:44.084698 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:44.084698 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:44.084698 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:44.084698 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:44.084698 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:44.085606 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:44.085606 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:44.085606 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:44.085606 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:44.085606 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:44.085606 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:44.085606 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:44.085606 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:44.085606 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:44.085606 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:44.085606 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 16:29:44.085606 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:29:44.085606 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 16:29:44.085606 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:44.089235 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 488/992] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:29:43 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:44.090663 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fff5e1-ecdf-4b45-ada0-c8a7ac377e13 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:44.187980 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:44.191577 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fff5e1-ecdf-4b45-ada0-c8a7ac377e13 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:44.225296 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298372fe-ce77-4d06-aa37-e56c6361e9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:44.225968 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298372fe-ce77-4d06-aa37-e56c6361e9bf None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:44.226254 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298372fe-ce77-4d06-aa37-e56c6361e9bf None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:44.226532 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298372fe-ce77-4d06-aa37-e56c6361e9bf None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:44.227248 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298372fe-ce77-4d06-aa37-e56c6361e9bf None admin] RBAC: Adding query filter params (name=compute) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:44.227558 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298372fe-ce77-4d06-aa37-e56c6361e9bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:44.228944 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298372fe-ce77-4d06-aa37-e56c6361e9bf None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:44.234788 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 505/993] 10.222.0.36 () {56 vars in 1112 bytes} [Thu May 19 16:29:44 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:44.240233 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690940e9-f362-470a-a812-0cfb1ed1b591 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:44.343519 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690940e9-f362-470a-a812-0cfb1ed1b591 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:44.378440 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf949e25-451b-48c0-b372-27719a6ebcfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:44.379115 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf949e25-451b-48c0-b372-27719a6ebcfc None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:44.379403 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf949e25-451b-48c0-b372-27719a6ebcfc None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:44.379684 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf949e25-451b-48c0-b372-27719a6ebcfc None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:44.380404 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf949e25-451b-48c0-b372-27719a6ebcfc None admin] RBAC: Adding query filter params (type=compute) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:44.380713 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf949e25-451b-48c0-b372-27719a6ebcfc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:44.382119 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf949e25-451b-48c0-b372-27719a6ebcfc None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:44.387548 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 489/994] 10.222.0.36 () {56 vars in 1112 bytes} [Thu May 19 16:29:44 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:29:44.393121 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298372fe-ce77-4d06-aa37-e56c6361e9bf None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:44.493544 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298372fe-ce77-4d06-aa37-e56c6361e9bf None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:44.528087 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f86bcf-5102-45e4-a53b-dc7ea99713cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:44.528749 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f86bcf-5102-45e4-a53b-dc7ea99713cd None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:44.529034 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f86bcf-5102-45e4-a53b-dc7ea99713cd None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:44.529337 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f86bcf-5102-45e4-a53b-dc7ea99713cd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:44.530671 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f86bcf-5102-45e4-a53b-dc7ea99713cd None admin] RBAC: Adding query filter params (interface=public, service_id=887f4e020b0e45bd888b40803baa6299, region_id=RegionOne) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:44.530988 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f86bcf-5102-45e4-a53b-dc7ea99713cd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:44.532377 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f86bcf-5102-45e4-a53b-dc7ea99713cd None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:44.537849 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 506/995] 10.222.0.36 () {56 vars in 1251 bytes} [Thu May 19 16:29:44 2022] GET /identity/v3/endpoints?service_id=887f4e020b0e45bd888b40803baa6299&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:44.543052 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf949e25-451b-48c0-b372-27719a6ebcfc None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:44.562096 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:44.649612 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf949e25-451b-48c0-b372-27719a6ebcfc None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:44.687591 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f969c462-9341-4f9c-b72c-bc9e4ae806fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:44.688276 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969c462-9341-4f9c-b72c-bc9e4ae806fa None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:44.688556 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969c462-9341-4f9c-b72c-bc9e4ae806fa None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:44.688840 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969c462-9341-4f9c-b72c-bc9e4ae806fa None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:44.689566 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f969c462-9341-4f9c-b72c-bc9e4ae806fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:44.690951 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f969c462-9341-4f9c-b72c-bc9e4ae806fa None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:44.705454 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 490/996] 10.222.0.36 () {56 vars in 1087 bytes} [Thu May 19 16:29:44 2022] GET /identity/v3/services => generated 1727 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:29:44.825924 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490bb6dd-cd75-4307-8578-979a6bfad8f8 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:44.826276 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490bb6dd-cd75-4307-8578-979a6bfad8f8 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:44.826493 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490bb6dd-cd75-4307-8578-979a6bfad8f8 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:44.828131 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 507/997] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:29:44.833940 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962ce0c-79e9-4b04-b7f5-afdf5fd28763 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:44.834479 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962ce0c-79e9-4b04-b7f5-afdf5fd28763 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:44.834479 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962ce0c-79e9-4b04-b7f5-afdf5fd28763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:44.852628 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-6962ce0c-79e9-4b04-b7f5-afdf5fd28763 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:44.853893 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-6962ce0c-79e9-4b04-b7f5-afdf5fd28763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:44.938574 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 491/998] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:44 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:29:44.960602 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490bb6dd-cd75-4307-8578-979a6bfad8f8 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:44.990502 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:45.068658 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490bb6dd-cd75-4307-8578-979a6bfad8f8 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:45.105557 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53301829-1b97-4f63-8e50-0bf336d42010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:45.106352 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53301829-1b97-4f63-8e50-0bf336d42010 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:45.106631 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53301829-1b97-4f63-8e50-0bf336d42010 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:45.106905 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53301829-1b97-4f63-8e50-0bf336d42010 None admin] PATH_INFO: `/v3/services/image` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:45.107561 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53301829-1b97-4f63-8e50-0bf336d42010 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:45.108988 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53301829-1b97-4f63-8e50-0bf336d42010 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:45.120705 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-53301829-1b97-4f63-8e50-0bf336d42010 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 16:29:45.120705 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:45.120705 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 16:29:45.120705 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 16:29:45.120705 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 16:29:45.120705 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 16:29:45.120705 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 16:29:45.120705 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:29:45.120705 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 16:29:45.120705 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:45.120705 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 16:29:45.120705 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:45.120705 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:45.120705 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:45.120705 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:45.120705 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:45.120705 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:45.121761 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:45.121761 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:45.121761 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:45.121761 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:45.121761 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:45.121761 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:45.121761 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 16:29:45.121761 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 16:29:45.121761 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 16:29:45.121761 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 16:29:45.121761 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:45.121761 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:45.121761 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:45.121761 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:45.121761 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:45.121761 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:45.122612 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:45.122612 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:45.122612 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:45.122612 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:45.122612 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:45.122612 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:45.122612 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:45.122612 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:45.122612 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:45.122612 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:45.122612 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 16:29:45.122612 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:29:45.122612 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 16:29:45.122612 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:45.123996 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 508/999] 10.222.0.36 () {56 vars in 1105 bytes} [Thu May 19 16:29:44 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:45.127893 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6962ce0c-79e9-4b04-b7f5-afdf5fd28763 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:45.140539 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:45.233930 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6962ce0c-79e9-4b04-b7f5-afdf5fd28763 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:45.270784 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b656bddb-5171-4f8a-af8e-aef3c110e875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:45.271455 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656bddb-5171-4f8a-af8e-aef3c110e875 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:45.271739 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656bddb-5171-4f8a-af8e-aef3c110e875 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:45.272014 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656bddb-5171-4f8a-af8e-aef3c110e875 None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:45.272729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b656bddb-5171-4f8a-af8e-aef3c110e875 None admin] RBAC: Adding query filter params (name=image) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:45.273067 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b656bddb-5171-4f8a-af8e-aef3c110e875 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:45.274445 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b656bddb-5171-4f8a-af8e-aef3c110e875 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:45.283851 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 492/1000] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:29:45 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:45.285118 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53301829-1b97-4f63-8e50-0bf336d42010 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:45.380579 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:45.390798 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53301829-1b97-4f63-8e50-0bf336d42010 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:45.414697 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea5b89d-47f5-43b7-8b0e-1e540c7b6de0 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:45.414992 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea5b89d-47f5-43b7-8b0e-1e540c7b6de0 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:45.415275 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea5b89d-47f5-43b7-8b0e-1e540c7b6de0 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:45.416663 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 493/1001] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:29:45.421686 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a6c4f-a5f6-432f-9ab4-6bc2f1164f9e None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:45.422053 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a6c4f-a5f6-432f-9ab4-6bc2f1164f9e None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:45.422261 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a6c4f-a5f6-432f-9ab4-6bc2f1164f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:45.425309 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731b7bac-bccc-495d-a2ad-58007436108e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:45.425986 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b7bac-bccc-495d-a2ad-58007436108e None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:45.426271 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b7bac-bccc-495d-a2ad-58007436108e None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:45.426547 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b7bac-bccc-495d-a2ad-58007436108e None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:45.427455 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731b7bac-bccc-495d-a2ad-58007436108e None admin] RBAC: Adding query filter params (type=image) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:45.427776 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731b7bac-bccc-495d-a2ad-58007436108e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:45.430017 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731b7bac-bccc-495d-a2ad-58007436108e None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:45.441410 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 509/1002] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:29:45 2022] GET /identity/v3/services?type=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:45.441605 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-605a6c4f-a5f6-432f-9ab4-6bc2f1164f9e None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:45.441605 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-605a6c4f-a5f6-432f-9ab4-6bc2f1164f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:45.525641 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 494/1003] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:45 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 16:29:45.543876 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731b7bac-bccc-495d-a2ad-58007436108e None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:45.659450 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731b7bac-bccc-495d-a2ad-58007436108e None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:45.694356 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85053f1-35a9-4afb-8a56-f3366f4d992d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:45.695320 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85053f1-35a9-4afb-8a56-f3366f4d992d None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:45.695857 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85053f1-35a9-4afb-8a56-f3366f4d992d None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:45.696272 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85053f1-35a9-4afb-8a56-f3366f4d992d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:45.697051 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85053f1-35a9-4afb-8a56-f3366f4d992d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:45.698596 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85053f1-35a9-4afb-8a56-f3366f4d992d None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:45.710482 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-c85053f1-35a9-4afb-8a56-f3366f4d992d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 16:29:45.710482 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:45.710482 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 16:29:45.710482 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 16:29:45.710482 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 16:29:45.710482 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 16:29:45.710482 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 16:29:45.710482 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:29:45.710482 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 16:29:45.710482 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:45.710482 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 16:29:45.710482 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:45.710482 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:45.710482 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:45.710482 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:45.710482 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:45.710482 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:45.711492 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:45.711492 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:45.711492 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:45.711492 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:45.711492 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:45.711492 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:45.711492 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 16:29:45.711492 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 16:29:45.711492 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 16:29:45.711492 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 16:29:45.711492 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:45.711492 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:45.711492 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:45.711492 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:45.711492 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:45.711492 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:45.712340 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:45.712340 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:45.712340 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:45.712340 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:45.712340 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:45.712340 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:45.712340 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:45.712340 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:45.712340 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:45.712340 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:45.712340 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 16:29:45.712340 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:29:45.712340 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 16:29:45.712340 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:45.712340 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 510/1004] 10.222.0.36 () {56 vars in 1126 bytes} [Thu May 19 16:29:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:45.719210 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a6c4f-a5f6-432f-9ab4-6bc2f1164f9e None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:45.722297 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:45.821379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a6c4f-a5f6-432f-9ab4-6bc2f1164f9e None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:45.857192 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f09b113-00df-48e7-9e7d-37e7ce6e5a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:45.857942 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09b113-00df-48e7-9e7d-37e7ce6e5a30 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:45.858677 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09b113-00df-48e7-9e7d-37e7ce6e5a30 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:45.858958 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09b113-00df-48e7-9e7d-37e7ce6e5a30 None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:45.859668 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f09b113-00df-48e7-9e7d-37e7ce6e5a30 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:45.859975 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f09b113-00df-48e7-9e7d-37e7ce6e5a30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:45.861400 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f09b113-00df-48e7-9e7d-37e7ce6e5a30 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:45.870705 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 495/1005] 10.222.0.36 () {56 vars in 1122 bytes} [Thu May 19 16:29:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:45.871892 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85053f1-35a9-4afb-8a56-f3366f4d992d None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:45.895932 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:45.979111 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85053f1-35a9-4afb-8a56-f3366f4d992d None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:46.014030 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef616766-228b-4f72-a28f-cb82a4c1f186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:46.015330 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef616766-228b-4f72-a28f-cb82a4c1f186 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:46.015620 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef616766-228b-4f72-a28f-cb82a4c1f186 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:46.015900 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef616766-228b-4f72-a28f-cb82a4c1f186 None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:46.016644 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef616766-228b-4f72-a28f-cb82a4c1f186 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:46.016953 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef616766-228b-4f72-a28f-cb82a4c1f186 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:46.018384 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef616766-228b-4f72-a28f-cb82a4c1f186 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:46.023809 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 511/1006] 10.222.0.36 () {56 vars in 1122 bytes} [Thu May 19 16:29:45 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:29:46.029281 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f09b113-00df-48e7-9e7d-37e7ce6e5a30 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:46.098812 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:46.133082 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f09b113-00df-48e7-9e7d-37e7ce6e5a30 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:46.168320 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e9f33-a404-483d-8423-c9981e9d76dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:46.168992 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e9f33-a404-483d-8423-c9981e9d76dd None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:46.169313 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e9f33-a404-483d-8423-c9981e9d76dd None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:46.169630 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e9f33-a404-483d-8423-c9981e9d76dd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:46.170393 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004e9f33-a404-483d-8423-c9981e9d76dd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:46.171805 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004e9f33-a404-483d-8423-c9981e9d76dd None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:46.187071 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 496/1007] 10.222.0.36 () {60 vars in 1144 bytes} [Thu May 19 16:29:46 2022] POST /identity/v3/endpoints => generated 325 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:29:46.948408 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05590bb-cabd-4036-8cbd-f78594d1b478 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:46.948951 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05590bb-cabd-4036-8cbd-f78594d1b478 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:46.948951 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05590bb-cabd-4036-8cbd-f78594d1b478 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:46.950523 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 512/1008] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:29:46.955992 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829b30a-8cd3-4e81-af09-b3a3eafd8e4a None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:46.956463 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829b30a-8cd3-4e81-af09-b3a3eafd8e4a None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:46.956542 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829b30a-8cd3-4e81-af09-b3a3eafd8e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:46.976268 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-8829b30a-8cd3-4e81-af09-b3a3eafd8e4a None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:46.977485 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-8829b30a-8cd3-4e81-af09-b3a3eafd8e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:47.056316 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 497/1009] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:46 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:29:47.078341 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05590bb-cabd-4036-8cbd-f78594d1b478 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:47.166881 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:47.184698 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05590bb-cabd-4036-8cbd-f78594d1b478 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:47.219092 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8009a84c-72a3-46f9-b0a0-659e6427f978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:47.219888 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8009a84c-72a3-46f9-b0a0-659e6427f978 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:47.220179 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8009a84c-72a3-46f9-b0a0-659e6427f978 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:47.220460 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8009a84c-72a3-46f9-b0a0-659e6427f978 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:47.221162 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8009a84c-72a3-46f9-b0a0-659e6427f978 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:47.222568 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8009a84c-72a3-46f9-b0a0-659e6427f978 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:47.234579 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-8009a84c-72a3-46f9-b0a0-659e6427f978 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 16:29:47.234579 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:47.234579 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 16:29:47.234579 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 16:29:47.234579 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 16:29:47.234579 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 16:29:47.234579 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 16:29:47.234579 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:29:47.234579 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 16:29:47.234579 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:47.234579 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 16:29:47.234579 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:47.234579 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:47.234579 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:47.234579 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:47.234579 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:47.234579 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:47.235619 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:47.235619 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:47.235619 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:47.235619 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:47.235619 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:47.235619 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:47.235619 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 16:29:47.235619 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 16:29:47.235619 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 16:29:47.235619 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 16:29:47.235619 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:47.235619 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:47.235619 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:47.235619 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:47.235619 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:47.235619 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:47.236474 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:47.236474 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:47.236474 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:47.236474 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:47.236474 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:47.236474 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:47.236474 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:47.236474 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:47.236474 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:47.236474 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:47.236474 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 16:29:47.236474 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:29:47.236474 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 16:29:47.236474 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:47.236474 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 513/1010] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:29:47 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:47.241856 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8829b30a-8cd3-4e81-af09-b3a3eafd8e4a None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:47.297607 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:47.345414 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8829b30a-8cd3-4e81-af09-b3a3eafd8e4a None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:47.381992 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cbbea6-f188-4a7b-bf13-4829ab6ea120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:47.382666 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbbea6-f188-4a7b-bf13-4829ab6ea120 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:47.382953 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbbea6-f188-4a7b-bf13-4829ab6ea120 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:47.383230 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbbea6-f188-4a7b-bf13-4829ab6ea120 None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:47.383953 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cbbea6-f188-4a7b-bf13-4829ab6ea120 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:47.384259 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cbbea6-f188-4a7b-bf13-4829ab6ea120 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:47.385683 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cbbea6-f188-4a7b-bf13-4829ab6ea120 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:47.391225 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 498/1011] 10.222.0.36 () {56 vars in 1112 bytes} [Thu May 19 16:29:47 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:47.397080 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8009a84c-72a3-46f9-b0a0-659e6427f978 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:47.504576 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8009a84c-72a3-46f9-b0a0-659e6427f978 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:47.539210 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e97ab-44b3-42d8-aa1d-5e483a043aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:47.539906 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e97ab-44b3-42d8-aa1d-5e483a043aa1 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:47.540193 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e97ab-44b3-42d8-aa1d-5e483a043aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:47.540471 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e97ab-44b3-42d8-aa1d-5e483a043aa1 None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:47.541236 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109e97ab-44b3-42d8-aa1d-5e483a043aa1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:47.541554 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109e97ab-44b3-42d8-aa1d-5e483a043aa1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:47.543005 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109e97ab-44b3-42d8-aa1d-5e483a043aa1 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:47.553767 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 514/1012] 10.222.0.36 () {56 vars in 1112 bytes} [Thu May 19 16:29:47 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:47.554885 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cbbea6-f188-4a7b-bf13-4829ab6ea120 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:47.658677 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cbbea6-f188-4a7b-bf13-4829ab6ea120 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:47.676515 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:47.693904 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ce0ea-dac1-457b-be63-6f24f414512f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:47.694570 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ce0ea-dac1-457b-be63-6f24f414512f None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:47.694859 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ce0ea-dac1-457b-be63-6f24f414512f None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:47.695141 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ce0ea-dac1-457b-be63-6f24f414512f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:47.695902 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ce0ea-dac1-457b-be63-6f24f414512f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:47.697338 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ce0ea-dac1-457b-be63-6f24f414512f None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:47.715775 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 499/1013] 10.222.0.36 () {60 vars in 1144 bytes} [Thu May 19 16:29:47 2022] POST /identity/v3/endpoints => generated 334 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:29:47.734498 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9626ba2d-073f-4a8f-9aa2-5c3e7ce81829 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:47.734801 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9626ba2d-073f-4a8f-9aa2-5c3e7ce81829 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:47.735223 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9626ba2d-073f-4a8f-9aa2-5c3e7ce81829 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:47.736794 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 515/1014] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:29:47.742834 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948f367-2667-400d-8081-ab96151eba59 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:47.743154 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948f367-2667-400d-8081-ab96151eba59 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:47.743411 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948f367-2667-400d-8081-ab96151eba59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:47.761355 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-4948f367-2667-400d-8081-ab96151eba59 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:47.762474 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-4948f367-2667-400d-8081-ab96151eba59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:47.845122 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 500/1015] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:47 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:29:47.867866 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9626ba2d-073f-4a8f-9aa2-5c3e7ce81829 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:47.885481 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:47.989210 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9626ba2d-073f-4a8f-9aa2-5c3e7ce81829 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:48.026799 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4fe450-3f53-4d78-8c0d-6b13f3ffa7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:48.027871 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4fe450-3f53-4d78-8c0d-6b13f3ffa7a4 None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:48.028364 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4fe450-3f53-4d78-8c0d-6b13f3ffa7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:48.028877 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4fe450-3f53-4d78-8c0d-6b13f3ffa7a4 None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:48.030409 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4fe450-3f53-4d78-8c0d-6b13f3ffa7a4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:48.031994 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4fe450-3f53-4d78-8c0d-6b13f3ffa7a4 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:48.048346 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 516/1016] 10.222.0.36 () {60 vars in 1141 bytes} [Thu May 19 16:29:47 2022] POST /identity/v3/services => generated 242 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:29:48.471706 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66580173-bc5d-43bd-af9b-4146f2d94bf7 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:48.472231 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66580173-bc5d-43bd-af9b-4146f2d94bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:48.472339 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66580173-bc5d-43bd-af9b-4146f2d94bf7 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:48.473948 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 501/1017] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:29:48.479224 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f78be-d52b-4a4f-b534-c433f70391f5 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:48.479586 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f78be-d52b-4a4f-b534-c433f70391f5 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:48.479808 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f78be-d52b-4a4f-b534-c433f70391f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:48.496719 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-419f78be-d52b-4a4f-b534-c433f70391f5 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:48.497946 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-419f78be-d52b-4a4f-b534-c433f70391f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:48.568695 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:48.581827 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 517/1018] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:48 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:29:48.600817 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66580173-bc5d-43bd-af9b-4146f2d94bf7 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:48.701508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:48.706557 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66580173-bc5d-43bd-af9b-4146f2d94bf7 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:48.741012 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ae7505-9c24-4ff0-8224-b9d97263643f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:48.741834 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ae7505-9c24-4ff0-8224-b9d97263643f None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:48.742123 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ae7505-9c24-4ff0-8224-b9d97263643f None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:48.742404 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ae7505-9c24-4ff0-8224-b9d97263643f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:48.743596 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ae7505-9c24-4ff0-8224-b9d97263643f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:48.745033 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ae7505-9c24-4ff0-8224-b9d97263643f None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:48.746859 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 502/1019] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:29:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:48.753097 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419f78be-d52b-4a4f-b534-c433f70391f5 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:48.851891 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419f78be-d52b-4a4f-b534-c433f70391f5 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:48.886421 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050211a1-7db0-4108-bbfd-c1d8ca2e4ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:48.891680 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050211a1-7db0-4108-bbfd-c1d8ca2e4ee4 None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:48.891680 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050211a1-7db0-4108-bbfd-c1d8ca2e4ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:48.891680 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050211a1-7db0-4108-bbfd-c1d8ca2e4ee4 None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:48.891680 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050211a1-7db0-4108-bbfd-c1d8ca2e4ee4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:48.891680 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050211a1-7db0-4108-bbfd-c1d8ca2e4ee4 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:48.925192 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 518/1020] 10.222.0.36 () {60 vars in 1141 bytes} [Thu May 19 16:29:48 2022] POST /identity/v3/projects => generated 311 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:29:50.060765 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb351cba-4ed7-4999-a219-5a2dfb1bd703 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:50.061333 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb351cba-4ed7-4999-a219-5a2dfb1bd703 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:50.061428 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb351cba-4ed7-4999-a219-5a2dfb1bd703 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:50.062924 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 503/1021] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:29:50.067968 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e4bd4-7319-410f-b484-06347af155bb None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:50.068392 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e4bd4-7319-410f-b484-06347af155bb None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:50.068794 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e4bd4-7319-410f-b484-06347af155bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:50.087234 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-bc4e4bd4-7319-410f-b484-06347af155bb None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:50.088160 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-bc4e4bd4-7319-410f-b484-06347af155bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:50.145919 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:50.169891 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 519/1022] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:50 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:29:50.192603 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb351cba-4ed7-4999-a219-5a2dfb1bd703 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:50.296552 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb351cba-4ed7-4999-a219-5a2dfb1bd703 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:50.296875 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee4401-40a4-46b9-af9c-a4923c2a5fa4 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:50.297092 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee4401-40a4-46b9-af9c-a4923c2a5fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:50.297402 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee4401-40a4-46b9-af9c-a4923c2a5fa4 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:50.298964 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 520/1023] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 16:29:50.303573 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f5eb0-e213-4e9d-8a12-f2909b2cef08 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:50.303859 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f5eb0-e213-4e9d-8a12-f2909b2cef08 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:50.304134 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f5eb0-e213-4e9d-8a12-f2909b2cef08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:50.321064 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-b88f5eb0-e213-4e9d-8a12-f2909b2cef08 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:50.322373 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b88f5eb0-e213-4e9d-8a12-f2909b2cef08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:50.333830 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c841e294-ad1f-44f0-b512-812d4bcb919e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:50.335222 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841e294-ad1f-44f0-b512-812d4bcb919e None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:50.335513 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841e294-ad1f-44f0-b512-812d4bcb919e None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:50.335792 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841e294-ad1f-44f0-b512-812d4bcb919e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:50.349356 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c841e294-ad1f-44f0-b512-812d4bcb919e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:29:50.359716 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-c841e294-ad1f-44f0-b512-812d4bcb919e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 16:29:50.359716 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:50.359716 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:50.359716 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:50.359716 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:50.359716 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:50.359716 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:50.359716 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:50.359716 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:50.359716 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:50.359716 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:50.359716 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:50.359716 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:29:50.359716 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:29:50.359716 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:29:50.359716 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:29:50.360650 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:29:50.360650 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:29:50.360650 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:29:50.360650 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:29:50.360650 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:50.360650 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:50.360650 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:50.360650 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:50.360650 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:50.360650 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:50.360650 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:50.360650 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:50.360650 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:50.360650 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:50.360650 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:50.360650 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:50.361995 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:50.361995 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:50.361995 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:50.361995 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:50.361995 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:29:50.361995 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:29:50.361995 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:29:50.361995 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:29:50.361995 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:29:50.361995 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:29:50.361995 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 16:29:50.361995 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:50.361995 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 504/1024] 10.222.0.36 () {56 vars in 1120 bytes} [Thu May 19 16:29:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:50.366702 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c841e294-ad1f-44f0-b512-812d4bcb919e None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:50.402763 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:50.404581 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 521/1025] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:50 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 16:29:50.429098 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88f5eb0-e213-4e9d-8a12-f2909b2cef08 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:50.486423 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c841e294-ad1f-44f0-b512-812d4bcb919e None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:50.524844 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f97f0c-97fb-4348-8d17-ff43a4a5cba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:50.525569 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f97f0c-97fb-4348-8d17-ff43a4a5cba0 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:50.525856 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f97f0c-97fb-4348-8d17-ff43a4a5cba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:50.526137 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f97f0c-97fb-4348-8d17-ff43a4a5cba0 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:50.526909 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f97f0c-97fb-4348-8d17-ff43a4a5cba0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:50.527202 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f97f0c-97fb-4348-8d17-ff43a4a5cba0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:50.528644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f97f0c-97fb-4348-8d17-ff43a4a5cba0 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:50.535087 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88f5eb0-e213-4e9d-8a12-f2909b2cef08 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:50.537014 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 505/1026] 10.222.0.36 () {56 vars in 1115 bytes} [Thu May 19 16:29:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:29:50.542324 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f97f0c-97fb-4348-8d17-ff43a4a5cba0 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:50.547267 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:50.574644 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec69fe4-d8f2-48f6-8c42-5a4bbc2711e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:50.575748 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec69fe4-d8f2-48f6-8c42-5a4bbc2711e4 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:50.576034 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec69fe4-d8f2-48f6-8c42-5a4bbc2711e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:50.576313 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec69fe4-d8f2-48f6-8c42-5a4bbc2711e4 None admin] PATH_INFO: `/v3/services/image` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:50.577550 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec69fe4-d8f2-48f6-8c42-5a4bbc2711e4 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:50.578940 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec69fe4-d8f2-48f6-8c42-5a4bbc2711e4 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:50.591747 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-7ec69fe4-d8f2-48f6-8c42-5a4bbc2711e4 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 16:29:50.591747 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:50.591747 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 16:29:50.591747 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 16:29:50.591747 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 16:29:50.591747 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 16:29:50.591747 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 16:29:50.591747 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:29:50.591747 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 16:29:50.591747 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:50.591747 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 16:29:50.591747 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:50.591747 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:50.591747 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:50.591747 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:50.591747 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:50.591747 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:50.593472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:50.593472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:50.593472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:50.593472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:50.593472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:50.593472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:50.593472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 16:29:50.593472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 16:29:50.593472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 16:29:50.593472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 16:29:50.593472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:50.593472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:50.593472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:50.593472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:50.593472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:50.593472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:50.594355 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:50.594355 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:50.594355 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:50.594355 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:50.594355 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:50.594355 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:50.594355 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:50.594355 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:50.594355 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:50.594355 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:50.594355 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 16:29:50.594355 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:29:50.594355 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 16:29:50.594355 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:50.594355 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 522/1027] 10.222.0.36 () {56 vars in 1105 bytes} [Thu May 19 16:29:50 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:50.599501 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec69fe4-d8f2-48f6-8c42-5a4bbc2711e4 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:50.645660 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:50.656904 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f97f0c-97fb-4348-8d17-ff43a4a5cba0 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:50.693517 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c07f8a7-bb1f-4fc2-9c75-7897082a850c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:50.694767 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07f8a7-bb1f-4fc2-9c75-7897082a850c None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:50.695233 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07f8a7-bb1f-4fc2-9c75-7897082a850c None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:50.695938 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07f8a7-bb1f-4fc2-9c75-7897082a850c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:50.709322 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c07f8a7-bb1f-4fc2-9c75-7897082a850c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:50.710203 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec69fe4-d8f2-48f6-8c42-5a4bbc2711e4 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:50.710786 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c07f8a7-bb1f-4fc2-9c75-7897082a850c None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:50.731124 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-7c07f8a7-bb1f-4fc2-9c75-7897082a850c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:50.731124 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:50.731124 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:50.731124 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:50.731124 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:50.731124 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:50.731124 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:50.731124 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:50.731124 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:50.731124 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:50.731124 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:50.731124 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:50.731124 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:29:50.731124 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:29:50.731124 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:29:50.732268 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:29:50.732268 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:50.732268 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:50.732268 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:50.732268 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:50.732268 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:50.732268 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:50.732268 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:50.732268 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:50.732268 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:50.732268 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:50.732268 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:50.732268 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:50.732268 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:50.732268 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:50.732268 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:50.732268 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:50.733777 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:29:50.733777 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:29:50.733777 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:50.733777 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:50.733777 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 506/1028] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:29:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 16:29:50.738345 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c07f8a7-bb1f-4fc2-9c75-7897082a850c None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:50.750909 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e46ca38-a67a-4395-97f2-f3a733068200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:50.751588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46ca38-a67a-4395-97f2-f3a733068200 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:50.751875 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46ca38-a67a-4395-97f2-f3a733068200 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:50.752154 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46ca38-a67a-4395-97f2-f3a733068200 None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:50.752873 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e46ca38-a67a-4395-97f2-f3a733068200 None admin] RBAC: Adding query filter params (name=image) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:50.753215 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e46ca38-a67a-4395-97f2-f3a733068200 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:50.754608 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e46ca38-a67a-4395-97f2-f3a733068200 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:50.759623 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 523/1029] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:29:50 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:29:50.765469 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e46ca38-a67a-4395-97f2-f3a733068200 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:50.808888 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:50.859292 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c07f8a7-bb1f-4fc2-9c75-7897082a850c None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:50.874954 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e46ca38-a67a-4395-97f2-f3a733068200 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:50.898292 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a95797-209f-42fb-9644-f4e4d659a182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:50.899152 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a95797-209f-42fb-9644-f4e4d659a182 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:50.899726 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a95797-209f-42fb-9644-f4e4d659a182 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:50.899860 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a95797-209f-42fb-9644-f4e4d659a182 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:50.900599 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a95797-209f-42fb-9644-f4e4d659a182 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:50.900910 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a95797-209f-42fb-9644-f4e4d659a182 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:50.902421 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a95797-209f-42fb-9644-f4e4d659a182 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:50.916097 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 507/1030] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:29:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:29:50.921395 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a95797-209f-42fb-9644-f4e4d659a182 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:50.923613 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e7cfff-aa71-4abd-bb42-b07640dd9fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:50.924294 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e7cfff-aa71-4abd-bb42-b07640dd9fb6 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:50.924579 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e7cfff-aa71-4abd-bb42-b07640dd9fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:50.924851 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e7cfff-aa71-4abd-bb42-b07640dd9fb6 None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:50.925615 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e7cfff-aa71-4abd-bb42-b07640dd9fb6 None admin] RBAC: Adding query filter params (type=image) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:50.925925 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e7cfff-aa71-4abd-bb42-b07640dd9fb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:50.927309 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e7cfff-aa71-4abd-bb42-b07640dd9fb6 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:50.929085 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:50.938677 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 524/1031] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:29:50 2022] GET /identity/v3/services?type=image => generated 350 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:29:50.939743 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e7cfff-aa71-4abd-bb42-b07640dd9fb6 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:51.045499 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a95797-209f-42fb-9644-f4e4d659a182 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:51.049017 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e7cfff-aa71-4abd-bb42-b07640dd9fb6 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:51.050941 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:51.084972 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4331086e-e332-4c17-a910-edf8ea277f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:51.085871 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331086e-e332-4c17-a910-edf8ea277f9e None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:51.086091 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331086e-e332-4c17-a910-edf8ea277f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:51.086369 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331086e-e332-4c17-a910-edf8ea277f9e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:51.090366 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a66fc6-e951-44c2-ab88-a87aab41135d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:51.091036 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a66fc6-e951-44c2-ab88-a87aab41135d None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:51.091321 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a66fc6-e951-44c2-ab88-a87aab41135d None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:51.091597 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a66fc6-e951-44c2-ab88-a87aab41135d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:51.093026 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a66fc6-e951-44c2-ab88-a87aab41135d None admin] RBAC: Adding query filter params (interface=public, service_id=ea4efa925b8f478a9783f2223c777d58, region_id=RegionOne) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:51.093703 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a66fc6-e951-44c2-ab88-a87aab41135d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:51.094246 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4331086e-e332-4c17-a910-edf8ea277f9e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:51.095196 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a66fc6-e951-44c2-ab88-a87aab41135d None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:51.095807 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4331086e-e332-4c17-a910-edf8ea277f9e None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:51.105361 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 525/1032] 10.222.0.36 () {56 vars in 1251 bytes} [Thu May 19 16:29:50 2022] GET /identity/v3/endpoints?service_id=ea4efa925b8f478a9783f2223c777d58&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:29:51.106372 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a66fc6-e951-44c2-ab88-a87aab41135d None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:51.111922 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-4331086e-e332-4c17-a910-edf8ea277f9e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 16:29:51.111922 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:51.111922 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:51.111922 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:51.111922 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:51.111922 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:51.111922 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:51.111922 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:51.111922 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:51.111922 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:51.111922 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:51.111922 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:51.111922 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 16:29:51.111922 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 16:29:51.111922 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 16:29:51.111922 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 16:29:51.112857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:51.112857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:51.112857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 16:29:51.112857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:29:51.112857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 16:29:51.112857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:29:51.112857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:51.112857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:51.112857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:51.112857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:51.112857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:51.112857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:51.112857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:51.112857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:51.112857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:51.112857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:51.112857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:51.112857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:51.113841 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:51.113841 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:51.113841 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 16:29:51.113841 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 16:29:51.113841 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 16:29:51.113841 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 16:29:51.113841 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 16:29:51.113841 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 16:29:51.113841 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 16:29:51.113841 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:51.113841 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 508/1033] 10.222.0.36 () {56 vars in 1093 bytes} [Thu May 19 16:29:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 16:29:51.120062 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4331086e-e332-4c17-a910-edf8ea277f9e None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:51.182160 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:51.223128 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4331086e-e332-4c17-a910-edf8ea277f9e None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:51.227371 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a66fc6-e951-44c2-ab88-a87aab41135d None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:51.258801 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f298f7b2-b1da-4ee1-a67e-596567861762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:51.259475 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298f7b2-b1da-4ee1-a67e-596567861762 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:51.259768 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298f7b2-b1da-4ee1-a67e-596567861762 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:51.260050 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298f7b2-b1da-4ee1-a67e-596567861762 None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:51.260996 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f298f7b2-b1da-4ee1-a67e-596567861762 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:51.261373 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f298f7b2-b1da-4ee1-a67e-596567861762 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:51.262865 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f298f7b2-b1da-4ee1-a67e-596567861762 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:51.266720 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc65eb2-6e4c-4183-ad5b-c4310edf2494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:51.267382 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc65eb2-6e4c-4183-ad5b-c4310edf2494 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:51.267672 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc65eb2-6e4c-4183-ad5b-c4310edf2494 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:51.267949 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc65eb2-6e4c-4183-ad5b-c4310edf2494 None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:51.268637 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc65eb2-6e4c-4183-ad5b-c4310edf2494 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:51.270318 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc65eb2-6e4c-4183-ad5b-c4310edf2494 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:51.278174 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 526/1034] 10.222.0.36 () {56 vars in 1087 bytes} [Thu May 19 16:29:51 2022] GET /identity/v3/services => generated 1957 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:29:51.279954 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d357484e-eba3-4de6-a030-07a3e350557b None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:51.280247 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d357484e-eba3-4de6-a030-07a3e350557b None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:51.280529 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d357484e-eba3-4de6-a030-07a3e350557b None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:51.284989 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 527/1035] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:29:51.290452 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 509/1036] 10.222.0.36 () {56 vars in 1133 bytes} [Thu May 19 16:29:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:51.291419 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb807bb-af99-4d97-aac8-6b4c33aef9e3 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:51.291714 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb807bb-af99-4d97-aac8-6b4c33aef9e3 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:51.291997 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb807bb-af99-4d97-aac8-6b4c33aef9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:51.296395 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f298f7b2-b1da-4ee1-a67e-596567861762 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:51.311480 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-9bb807bb-af99-4d97-aac8-6b4c33aef9e3 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:51.312719 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-9bb807bb-af99-4d97-aac8-6b4c33aef9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:51.341531 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:51.398207 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 528/1037] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:51 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 16:29:51.410129 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f298f7b2-b1da-4ee1-a67e-596567861762 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:51.422820 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb807bb-af99-4d97-aac8-6b4c33aef9e3 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:51.437886 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:51.457893 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e260d4-ddbc-4490-a559-b57f8d726595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:51.458562 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e260d4-ddbc-4490-a559-b57f8d726595 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:51.458846 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e260d4-ddbc-4490-a559-b57f8d726595 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:51.459122 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e260d4-ddbc-4490-a559-b57f8d726595 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:51.535817 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb807bb-af99-4d97-aac8-6b4c33aef9e3 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:51.539634 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e260d4-ddbc-4490-a559-b57f8d726595 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:51.541091 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e260d4-ddbc-4490-a559-b57f8d726595 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:51.574266 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517eb797-4ff9-456b-af46-f7f60247ce8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:51.575070 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517eb797-4ff9-456b-af46-f7f60247ce8b None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:51.575353 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517eb797-4ff9-456b-af46-f7f60247ce8b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:51.575628 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517eb797-4ff9-456b-af46-f7f60247ce8b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:51.577404 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517eb797-4ff9-456b-af46-f7f60247ce8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:51.578854 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517eb797-4ff9-456b-af46-f7f60247ce8b None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:51.580493 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 529/1038] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:29:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:29:51.586736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517eb797-4ff9-456b-af46-f7f60247ce8b None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:51.619516 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:51.652991 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 510/1039] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:29:51 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 16:29:51.654211 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e260d4-ddbc-4490-a559-b57f8d726595 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:51.677080 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:51.702387 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517eb797-4ff9-456b-af46-f7f60247ce8b None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:51.741378 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dbeedd-972e-4f31-a7fa-a2cfbabbc50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:51.742170 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dbeedd-972e-4f31-a7fa-a2cfbabbc50c None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:51.742450 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dbeedd-972e-4f31-a7fa-a2cfbabbc50c None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:51.742753 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dbeedd-972e-4f31-a7fa-a2cfbabbc50c None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:51.743435 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dbeedd-972e-4f31-a7fa-a2cfbabbc50c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:51.744884 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dbeedd-972e-4f31-a7fa-a2cfbabbc50c None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:51.765193 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e260d4-ddbc-4490-a559-b57f8d726595 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:51.769328 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 530/1040] 10.222.0.36 () {60 vars in 1132 bytes} [Thu May 19 16:29:51 2022] POST /identity/v3/users => generated 285 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:29:51.811777 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4cf28-929f-4a83-a7d9-bcf3f91a09f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:51.812663 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4cf28-929f-4a83-a7d9-bcf3f91a09f4 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:51.812937 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4cf28-929f-4a83-a7d9-bcf3f91a09f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:51.813290 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4cf28-929f-4a83-a7d9-bcf3f91a09f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:51.825299 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a4cf28-929f-4a83-a7d9-bcf3f91a09f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:51.827531 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a4cf28-929f-4a83-a7d9-bcf3f91a09f4 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:51.848428 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-42a4cf28-929f-4a83-a7d9-bcf3f91a09f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:51.848428 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:51.848428 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:51.848428 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:51.848428 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:51.848428 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:51.848428 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:51.848428 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:51.848428 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:51.848428 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:51.848428 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:51.848428 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:51.848428 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:29:51.848428 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:29:51.848428 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:29:51.849786 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:29:51.849786 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:51.849786 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:51.849786 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:51.849786 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:51.849786 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:51.849786 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:51.849786 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:51.849786 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:51.849786 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:51.849786 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:51.849786 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:51.849786 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:51.849786 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:51.849786 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:51.849786 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:51.849786 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:51.850755 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:29:51.850755 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:29:51.850755 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:51.850755 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:51.853530 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 511/1041] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:29:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:51.855705 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dbeedd-972e-4f31-a7fa-a2cfbabbc50c None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:51.961736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:51.981081 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dbeedd-972e-4f31-a7fa-a2cfbabbc50c None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:52.018339 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6351e714-fd83-4916-84fd-f27db17a01d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:52.019275 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6351e714-fd83-4916-84fd-f27db17a01d8 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:52.019551 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6351e714-fd83-4916-84fd-f27db17a01d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:52.019825 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6351e714-fd83-4916-84fd-f27db17a01d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:52.020549 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6351e714-fd83-4916-84fd-f27db17a01d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:52.020865 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6351e714-fd83-4916-84fd-f27db17a01d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:52.022280 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6351e714-fd83-4916-84fd-f27db17a01d8 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:52.035137 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 531/1042] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:29:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:52.041790 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4cf28-929f-4a83-a7d9-bcf3f91a09f4 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:52.116998 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:52.150410 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4cf28-929f-4a83-a7d9-bcf3f91a09f4 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:52.183977 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc162890-f1e9-45e7-9d31-be9d709b3577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:52.184776 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc162890-f1e9-45e7-9d31-be9d709b3577 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:52.185061 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc162890-f1e9-45e7-9d31-be9d709b3577 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:52.185367 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc162890-f1e9-45e7-9d31-be9d709b3577 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:52.192678 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc162890-f1e9-45e7-9d31-be9d709b3577 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:52.194140 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc162890-f1e9-45e7-9d31-be9d709b3577 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:52.208161 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-dc162890-f1e9-45e7-9d31-be9d709b3577 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:29:52.208161 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:52.208161 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:52.208161 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:52.208161 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:52.208161 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:52.208161 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:52.208161 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:52.208161 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:52.208161 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:52.208161 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:52.208161 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:52.208161 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 16:29:52.208161 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 16:29:52.208161 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 16:29:52.209060 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 16:29:52.209060 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:52.209060 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:52.209060 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:52.209060 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:52.209060 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:52.209060 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:52.209060 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:52.209060 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:52.209060 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:52.209060 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:52.209060 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:52.209060 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:52.209060 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:52.209060 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:52.209060 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:52.209060 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:52.209994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 16:29:52.209994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 16:29:52.209994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 16:29:52.209994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 16:29:52.209994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 16:29:52.209994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 16:29:52.209994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:29:52.209994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:52.213265 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 512/1043] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:29:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 16:29:52.215384 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6351e714-fd83-4916-84fd-f27db17a01d8 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:52.324989 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6351e714-fd83-4916-84fd-f27db17a01d8 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:52.360033 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe51c6e-6fba-4716-ae0a-35f6c4d388d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:52.360700 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe51c6e-6fba-4716-ae0a-35f6c4d388d8 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:52.360987 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe51c6e-6fba-4716-ae0a-35f6c4d388d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:52.361304 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe51c6e-6fba-4716-ae0a-35f6c4d388d8 None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:52.362102 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe51c6e-6fba-4716-ae0a-35f6c4d388d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:52.362414 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe51c6e-6fba-4716-ae0a-35f6c4d388d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:52.363861 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe51c6e-6fba-4716-ae0a-35f6c4d388d8 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:52.381043 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 532/1044] 10.222.0.36 () {56 vars in 1148 bytes} [Thu May 19 16:29:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:52.382862 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc162890-f1e9-45e7-9d31-be9d709b3577 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:52.489723 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc162890-f1e9-45e7-9d31-be9d709b3577 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:52.520929 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:52.526696 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e825af1c-4384-41b1-bccf-8f1dcafc82bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:52.527356 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825af1c-4384-41b1-bccf-8f1dcafc82bd None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:52.527638 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825af1c-4384-41b1-bccf-8f1dcafc82bd None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:52.527913 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825af1c-4384-41b1-bccf-8f1dcafc82bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:52.528746 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e825af1c-4384-41b1-bccf-8f1dcafc82bd None admin] RBAC: Adding query filter params (role.id=58a72d6ac3ff4e278e79059c4fe2ec5a, scope.project.id=631c76794a124d6da22c626e590e4352, user.id=58bdeada00b74b7b8e186e6dc51e6dfb) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:52.529057 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e825af1c-4384-41b1-bccf-8f1dcafc82bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:52.531116 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e825af1c-4384-41b1-bccf-8f1dcafc82bd None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:52.545391 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 513/1045] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:29:52 2022] GET /identity/v3/role_assignments?user.id=58bdeada00b74b7b8e186e6dc51e6dfb&scope.project.id=631c76794a124d6da22c626e590e4352&role.id=58a72d6ac3ff4e278e79059c4fe2ec5a => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:53.533085 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2755f4-7bc9-4e3c-b9d7-ffe4b14ee52e None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:53.533662 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2755f4-7bc9-4e3c-b9d7-ffe4b14ee52e None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:53.533662 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2755f4-7bc9-4e3c-b9d7-ffe4b14ee52e None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:53.535207 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 533/1046] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:29:53.540262 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b676efe-89c0-47f2-888b-374786672aff None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:53.540694 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b676efe-89c0-47f2-888b-374786672aff None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:53.540846 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b676efe-89c0-47f2-888b-374786672aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:53.558309 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-6b676efe-89c0-47f2-888b-374786672aff None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:53.559487 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-6b676efe-89c0-47f2-888b-374786672aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:53.626623 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 514/1047] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:53 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:29:53.649528 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2755f4-7bc9-4e3c-b9d7-ffe4b14ee52e None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:53.653381 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:53.781942 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2755f4-7bc9-4e3c-b9d7-ffe4b14ee52e None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:53.816906 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e51e0d-f9a7-4d63-8776-8128787fde9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:53.817644 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e51e0d-f9a7-4d63-8776-8128787fde9b None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:53.817928 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e51e0d-f9a7-4d63-8776-8128787fde9b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:53.818202 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e51e0d-f9a7-4d63-8776-8128787fde9b None admin] PATH_INFO: `/v3/services/image` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:53.818911 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e51e0d-f9a7-4d63-8776-8128787fde9b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:53.820288 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e51e0d-f9a7-4d63-8776-8128787fde9b None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:53.832311 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-14e51e0d-f9a7-4d63-8776-8128787fde9b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 16:29:53.832311 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:53.832311 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 16:29:53.832311 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 16:29:53.832311 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 16:29:53.832311 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 16:29:53.832311 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 16:29:53.832311 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:29:53.832311 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 16:29:53.832311 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:53.832311 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 16:29:53.832311 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:53.832311 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:53.832311 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:53.832311 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:53.832311 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:53.832311 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:53.833570 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:53.833570 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:53.833570 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:53.833570 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:53.833570 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:53.833570 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:53.833570 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 16:29:53.833570 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 16:29:53.833570 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 16:29:53.833570 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 16:29:53.833570 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:53.833570 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:53.833570 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:53.833570 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:53.833570 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:53.833570 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:53.834459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:53.834459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:53.834459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:53.834459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:53.834459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:53.834459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:53.834459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:53.834459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:53.834459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:53.834459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:53.834459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 16:29:53.834459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:29:53.834459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 16:29:53.834459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:53.837542 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 534/1048] 10.222.0.36 () {56 vars in 1105 bytes} [Thu May 19 16:29:53 2022] GET /identity/v3/services/image => generated 86 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:53.839507 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b676efe-89c0-47f2-888b-374786672aff None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:53.972372 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b676efe-89c0-47f2-888b-374786672aff None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:53.997915 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:54.007502 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe3193-ee7a-4cec-963a-2fec728ee910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:54.008178 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe3193-ee7a-4cec-963a-2fec728ee910 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:54.008461 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe3193-ee7a-4cec-963a-2fec728ee910 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:54.008784 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe3193-ee7a-4cec-963a-2fec728ee910 None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:54.009527 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafe3193-ee7a-4cec-963a-2fec728ee910 None admin] RBAC: Adding query filter params (name=image) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:54.009836 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafe3193-ee7a-4cec-963a-2fec728ee910 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:54.011196 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafe3193-ee7a-4cec-963a-2fec728ee910 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:54.021323 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 515/1049] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:29:53 2022] GET /identity/v3/services?name=image => generated 122 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:54.022555 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e51e0d-f9a7-4d63-8776-8128787fde9b None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:54.096571 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:54.126603 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528801b-82c8-4ef4-96e7-eafbf76cb780 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:54.126906 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528801b-82c8-4ef4-96e7-eafbf76cb780 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:54.127179 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528801b-82c8-4ef4-96e7-eafbf76cb780 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:54.128761 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 516/1050] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 16:29:54.133481 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0eebc6-4abd-417f-ba5b-8aa8329b4eaa None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:54.133767 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0eebc6-4abd-417f-ba5b-8aa8329b4eaa None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:54.133997 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e51e0d-f9a7-4d63-8776-8128787fde9b None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:54.134105 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0eebc6-4abd-417f-ba5b-8aa8329b4eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:54.153823 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-cf0eebc6-4abd-417f-ba5b-8aa8329b4eaa None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:54.155495 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-cf0eebc6-4abd-417f-ba5b-8aa8329b4eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:54.172404 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427bcb3f-7d65-43b7-a629-1230f717a2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:54.173576 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427bcb3f-7d65-43b7-a629-1230f717a2ab None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:54.174026 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427bcb3f-7d65-43b7-a629-1230f717a2ab None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:54.174309 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427bcb3f-7d65-43b7-a629-1230f717a2ab None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:54.175040 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427bcb3f-7d65-43b7-a629-1230f717a2ab None admin] RBAC: Adding query filter params (type=image) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:54.175478 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427bcb3f-7d65-43b7-a629-1230f717a2ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:54.176874 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427bcb3f-7d65-43b7-a629-1230f717a2ab None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:54.182313 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 535/1051] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:29:54 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:54.188286 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427bcb3f-7d65-43b7-a629-1230f717a2ab None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:54.230379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 517/1052] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:54 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 16:29:54.253222 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0eebc6-4abd-417f-ba5b-8aa8329b4eaa None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:54.294969 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427bcb3f-7d65-43b7-a629-1230f717a2ab None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:54.327226 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:54.333611 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca608072-5318-445f-9003-5579666be78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:54.334274 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca608072-5318-445f-9003-5579666be78c None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:54.334558 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca608072-5318-445f-9003-5579666be78c None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:54.334832 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca608072-5318-445f-9003-5579666be78c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:54.335586 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca608072-5318-445f-9003-5579666be78c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:54.336985 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca608072-5318-445f-9003-5579666be78c None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:54.365268 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 536/1053] 10.222.0.36 () {60 vars in 1144 bytes} [Thu May 19 16:29:54 2022] POST /identity/v3/endpoints => generated 327 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:29:54.365474 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0eebc6-4abd-417f-ba5b-8aa8329b4eaa None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:54.389975 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:54.413729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753df711-a374-4612-9a1d-e22804df8cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:54.414541 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753df711-a374-4612-9a1d-e22804df8cec None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:54.414827 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753df711-a374-4612-9a1d-e22804df8cec None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:54.415104 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753df711-a374-4612-9a1d-e22804df8cec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:54.427609 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753df711-a374-4612-9a1d-e22804df8cec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:29:54.436379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-753df711-a374-4612-9a1d-e22804df8cec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:29:54.436379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:54.436379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:54.436379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:54.436379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:54.436379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:54.436379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:54.436379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:54.436379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:54.436379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:54.436379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:54.436379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:54.436379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:29:54.436379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:29:54.436379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:29:54.436379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:29:54.437987 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:29:54.437987 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:29:54.437987 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:29:54.437987 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:29:54.437987 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:54.437987 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:54.437987 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:54.437987 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:54.437987 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:54.437987 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:54.437987 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:54.437987 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:54.437987 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:54.437987 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:54.437987 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:54.437987 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:54.440034 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:54.440034 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:54.440034 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:54.440034 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:54.440034 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:29:54.440034 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:29:54.440034 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:29:54.440034 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:29:54.440034 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:29:54.440034 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:29:54.440034 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:29:54.440034 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:54.441405 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 518/1054] 10.222.0.36 () {56 vars in 1096 bytes} [Thu May 19 16:29:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 16:29:54.443878 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca608072-5318-445f-9003-5579666be78c None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:54.567169 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca608072-5318-445f-9003-5579666be78c None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:54.606203 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbcab6b-94b8-47f7-b64d-7fbad908dc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:54.607284 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbcab6b-94b8-47f7-b64d-7fbad908dc6d None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:54.607626 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbcab6b-94b8-47f7-b64d-7fbad908dc6d None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:54.608490 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbcab6b-94b8-47f7-b64d-7fbad908dc6d None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:54.609302 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbcab6b-94b8-47f7-b64d-7fbad908dc6d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:54.609622 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbcab6b-94b8-47f7-b64d-7fbad908dc6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:54.611094 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbcab6b-94b8-47f7-b64d-7fbad908dc6d None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:54.620737 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 537/1055] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:29:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:54.626380 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753df711-a374-4612-9a1d-e22804df8cec None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:54.736823 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753df711-a374-4612-9a1d-e22804df8cec None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:54.760704 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0aaf9-114d-4bbe-b7bb-67ef78480295 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:54.761092 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0aaf9-114d-4bbe-b7bb-67ef78480295 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:54.761633 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0aaf9-114d-4bbe-b7bb-67ef78480295 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:54.763125 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 538/1056] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:29:54.770500 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f3a542-a53b-42e6-a16c-aff5a4b68096 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:54.770818 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f3a542-a53b-42e6-a16c-aff5a4b68096 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:54.771160 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f3a542-a53b-42e6-a16c-aff5a4b68096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:54.783100 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2827c997-be65-49d2-9733-ee3545c017cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:54.784714 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827c997-be65-49d2-9733-ee3545c017cb None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:54.785198 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827c997-be65-49d2-9733-ee3545c017cb None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:54.785500 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827c997-be65-49d2-9733-ee3545c017cb None admin] PATH_INFO: `/v3/users/c52e6c529cd34730ae07a8e890f337a6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:54.790250 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-34f3a542-a53b-42e6-a16c-aff5a4b68096 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:54.791508 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-34f3a542-a53b-42e6-a16c-aff5a4b68096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:54.800987 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2827c997-be65-49d2-9733-ee3545c017cb None admin] RBAC: Authorizing `identity:get_user(user_id=c52e6c529cd34730ae07a8e890f337a6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:54.802516 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2827c997-be65-49d2-9733-ee3545c017cb None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:54.810537 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 519/1057] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:29:54 2022] GET /identity/v3/users/c52e6c529cd34730ae07a8e890f337a6 => generated 285 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:54.810537 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2827c997-be65-49d2-9733-ee3545c017cb None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:54.847086 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:54.855061 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:54.881204 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 539/1058] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:29:54.903016 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f3a542-a53b-42e6-a16c-aff5a4b68096 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:54.936322 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2827c997-be65-49d2-9733-ee3545c017cb None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:54.973041 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33043c4e-9cb7-4505-8247-928caa2de670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:54.973714 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33043c4e-9cb7-4505-8247-928caa2de670 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:54.973999 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33043c4e-9cb7-4505-8247-928caa2de670 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:54.974277 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33043c4e-9cb7-4505-8247-928caa2de670 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:55.021802 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f3a542-a53b-42e6-a16c-aff5a4b68096 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:55.048435 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33043c4e-9cb7-4505-8247-928caa2de670 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:55.050594 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33043c4e-9cb7-4505-8247-928caa2de670 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:55.058139 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6d85e-90bf-4da1-9081-ab93962250bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:55.058951 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6d85e-90bf-4da1-9081-ab93962250bf None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:55.059237 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6d85e-90bf-4da1-9081-ab93962250bf None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:55.059512 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6d85e-90bf-4da1-9081-ab93962250bf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:55.071216 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac6d85e-90bf-4da1-9081-ab93962250bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:29:55.079994 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-6ac6d85e-90bf-4da1-9081-ab93962250bf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 16:29:55.079994 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:55.079994 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:55.079994 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:55.079994 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:55.079994 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:55.079994 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:55.079994 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:55.079994 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:55.079994 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:55.079994 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:55.079994 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:55.079994 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:29:55.079994 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:29:55.079994 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:29:55.079994 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:29:55.080953 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:29:55.080953 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:29:55.080953 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:29:55.080953 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:29:55.080953 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:55.080953 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:55.080953 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:55.080953 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:55.080953 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:55.080953 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:55.080953 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:55.080953 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:55.080953 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:55.080953 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:55.080953 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:55.080953 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:55.082438 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:55.082438 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:55.082438 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:55.082438 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:55.082438 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:29:55.082438 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:29:55.082438 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:29:55.082438 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:29:55.082438 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:29:55.082438 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:29:55.082438 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 16:29:55.082438 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:55.082438 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 540/1059] 10.222.0.36 () {56 vars in 1120 bytes} [Thu May 19 16:29:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 16:29:55.087109 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6d85e-90bf-4da1-9081-ab93962250bf None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:55.089270 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:55.142550 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:55.188417 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 520/1060] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:29:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 16:29:55.189646 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33043c4e-9cb7-4505-8247-928caa2de670 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:55.215741 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6d85e-90bf-4da1-9081-ab93962250bf None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:55.250719 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad47d4c-1956-49da-9a04-03d662431cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:55.251404 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad47d4c-1956-49da-9a04-03d662431cc5 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:55.251693 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad47d4c-1956-49da-9a04-03d662431cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:55.251975 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad47d4c-1956-49da-9a04-03d662431cc5 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:55.252728 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad47d4c-1956-49da-9a04-03d662431cc5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:55.253041 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad47d4c-1956-49da-9a04-03d662431cc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:55.254464 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad47d4c-1956-49da-9a04-03d662431cc5 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:55.263016 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 541/1061] 10.222.0.36 () {56 vars in 1115 bytes} [Thu May 19 16:29:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:29:55.270188 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad47d4c-1956-49da-9a04-03d662431cc5 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:55.299527 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33043c4e-9cb7-4505-8247-928caa2de670 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:55.336090 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94021feb-dd22-44a5-9093-ced6f7c63940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:55.336951 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94021feb-dd22-44a5-9093-ced6f7c63940 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:55.337273 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94021feb-dd22-44a5-9093-ced6f7c63940 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:55.337559 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94021feb-dd22-44a5-9093-ced6f7c63940 None admin] PATH_INFO: `/v3/projects/abba9b53b10946799633b36c9dee5f73` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:55.338767 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94021feb-dd22-44a5-9093-ced6f7c63940 None admin] RBAC: Authorizing `identity:get_project(project_id=abba9b53b10946799633b36c9dee5f73)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:55.340215 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94021feb-dd22-44a5-9093-ced6f7c63940 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:55.341957 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 521/1062] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:29:55 2022] GET /identity/v3/projects/abba9b53b10946799633b36c9dee5f73 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:55.347950 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94021feb-dd22-44a5-9093-ced6f7c63940 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:55.350753 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:55.376894 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad47d4c-1956-49da-9a04-03d662431cc5 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:55.378706 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:55.414571 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b628ea43-45a4-4eda-ba8c-cf6f4ab8b9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:55.415292 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b628ea43-45a4-4eda-ba8c-cf6f4ab8b9b0 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:55.415583 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b628ea43-45a4-4eda-ba8c-cf6f4ab8b9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:55.415959 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b628ea43-45a4-4eda-ba8c-cf6f4ab8b9b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:55.428672 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b628ea43-45a4-4eda-ba8c-cf6f4ab8b9b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:55.430184 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b628ea43-45a4-4eda-ba8c-cf6f4ab8b9b0 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:55.448344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-b628ea43-45a4-4eda-ba8c-cf6f4ab8b9b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:55.448344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:55.448344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:55.448344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:55.448344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:55.448344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:55.448344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:55.448344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:55.448344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:55.448344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:55.448344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:55.448344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:55.448344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:29:55.448344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:29:55.448344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:29:55.449318 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:29:55.449318 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:55.449318 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:55.449318 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:55.449318 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:55.449318 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:55.449318 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:55.449318 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:55.449318 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:55.449318 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:55.449318 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:55.449318 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:55.449318 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:55.449318 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:55.449318 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:55.449318 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:55.449318 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:55.450231 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:29:55.450231 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:29:55.450231 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:55.450231 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:55.450231 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 542/1063] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:29:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 16:29:55.455269 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b628ea43-45a4-4eda-ba8c-cf6f4ab8b9b0 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:55.457778 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94021feb-dd22-44a5-9093-ced6f7c63940 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:55.495128 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6827c09-7933-413a-82d5-06ee5560532a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:55.496210 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6827c09-7933-413a-82d5-06ee5560532a None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:55.496487 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6827c09-7933-413a-82d5-06ee5560532a None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:55.496770 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6827c09-7933-413a-82d5-06ee5560532a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:55.497663 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6827c09-7933-413a-82d5-06ee5560532a None admin] RBAC: Adding query filter params (role.id=e2915fe109244a07a9be96bd2b219fdd, scope.project.id=abba9b53b10946799633b36c9dee5f73, user.id=c52e6c529cd34730ae07a8e890f337a6) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:55.497981 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6827c09-7933-413a-82d5-06ee5560532a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:55.500019 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6827c09-7933-413a-82d5-06ee5560532a None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:55.517182 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 522/1064] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:29:55 2022] GET /identity/v3/role_assignments?user.id=c52e6c529cd34730ae07a8e890f337a6&scope.project.id=abba9b53b10946799633b36c9dee5f73&role.id=e2915fe109244a07a9be96bd2b219fdd => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:29:55.574227 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b628ea43-45a4-4eda-ba8c-cf6f4ab8b9b0 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:55.612601 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca6b9d7-ce4e-4056-a09b-2c8f3b7e3b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:55.613323 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6b9d7-ce4e-4056-a09b-2c8f3b7e3b5c None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:55.613615 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6b9d7-ce4e-4056-a09b-2c8f3b7e3b5c None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:55.613890 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6b9d7-ce4e-4056-a09b-2c8f3b7e3b5c None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:55.614637 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca6b9d7-ce4e-4056-a09b-2c8f3b7e3b5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:55.614952 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca6b9d7-ce4e-4056-a09b-2c8f3b7e3b5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:55.616372 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca6b9d7-ce4e-4056-a09b-2c8f3b7e3b5c None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:55.634821 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 543/1065] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:29:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:29:55.636421 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6827c09-7933-413a-82d5-06ee5560532a None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:55.718215 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:55.754791 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6827c09-7933-413a-82d5-06ee5560532a None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:55.788688 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3073e-478f-4b7d-bb06-ccff560a7ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:55.789527 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3073e-478f-4b7d-bb06-ccff560a7ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:55.789813 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3073e-478f-4b7d-bb06-ccff560a7ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:55.790096 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3073e-478f-4b7d-bb06-ccff560a7ea4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:55.798599 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c3073e-478f-4b7d-bb06-ccff560a7ea4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:55.800071 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c3073e-478f-4b7d-bb06-ccff560a7ea4 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:55.814292 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-26c3073e-478f-4b7d-bb06-ccff560a7ea4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 16:29:55.814292 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:55.814292 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:55.814292 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:55.814292 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:55.814292 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:55.814292 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:55.814292 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:55.814292 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:55.814292 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:55.814292 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:55.814292 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:55.814292 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 16:29:55.814292 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 16:29:55.814292 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 16:29:55.814292 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 16:29:55.815264 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:55.815264 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:55.815264 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 16:29:55.815264 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:29:55.815264 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 16:29:55.815264 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:29:55.815264 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:55.815264 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:55.815264 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:55.815264 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:55.815264 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:55.815264 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:55.815264 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:55.815264 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:55.815264 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:55.815264 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:55.815264 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:55.815264 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:55.816536 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:55.816536 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:55.816536 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 16:29:55.816536 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 16:29:55.816536 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 16:29:55.816536 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 16:29:55.816536 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 16:29:55.816536 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 16:29:55.816536 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 16:29:55.816536 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:55.816536 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 523/1066] 10.222.0.36 () {56 vars in 1093 bytes} [Thu May 19 16:29:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:55.821113 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca6b9d7-ce4e-4056-a09b-2c8f3b7e3b5c None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:55.839781 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:55.936063 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca6b9d7-ce4e-4056-a09b-2c8f3b7e3b5c None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:55.971037 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d642c01-b739-4b43-93b1-5e01948851d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:55.971632 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d642c01-b739-4b43-93b1-5e01948851d8 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:55.971915 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d642c01-b739-4b43-93b1-5e01948851d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:55.972190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d642c01-b739-4b43-93b1-5e01948851d8 None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:55.973029 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d642c01-b739-4b43-93b1-5e01948851d8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:55.973375 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d642c01-b739-4b43-93b1-5e01948851d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:55.974816 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d642c01-b739-4b43-93b1-5e01948851d8 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:55.997447 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 544/1067] 10.222.0.36 () {56 vars in 1133 bytes} [Thu May 19 16:29:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:29:55.999191 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3073e-478f-4b7d-bb06-ccff560a7ea4 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:56.103410 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3073e-478f-4b7d-bb06-ccff560a7ea4 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:56.129292 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:56.141708 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b49f3-54d0-40fd-9cf6-5daca1001f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:56.142510 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b49f3-54d0-40fd-9cf6-5daca1001f89 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:56.142793 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b49f3-54d0-40fd-9cf6-5daca1001f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:56.143069 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b49f3-54d0-40fd-9cf6-5daca1001f89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:56.155144 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0b49f3-54d0-40fd-9cf6-5daca1001f89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:56.156602 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0b49f3-54d0-40fd-9cf6-5daca1001f89 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:56.173742 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-ff0b49f3-54d0-40fd-9cf6-5daca1001f89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:56.173742 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:56.173742 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:56.173742 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:56.173742 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:56.173742 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:56.173742 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:56.173742 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:56.173742 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:56.173742 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:56.173742 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:56.173742 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:56.173742 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:29:56.173742 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:29:56.173742 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:29:56.174686 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:29:56.174686 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:56.174686 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:56.174686 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:56.174686 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:56.174686 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:56.174686 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:56.174686 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:56.174686 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:56.174686 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:56.174686 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:56.174686 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:56.174686 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:56.174686 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:56.174686 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:56.174686 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:56.174686 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:56.175618 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:29:56.175618 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:29:56.175618 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:56.175618 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:56.179599 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 524/1068] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:29:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:56.181003 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d642c01-b739-4b43-93b1-5e01948851d8 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:56.251622 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:56.283472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d642c01-b739-4b43-93b1-5e01948851d8 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:56.317709 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c55eba-66ce-4394-9329-3e52f47424c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:56.318381 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c55eba-66ce-4394-9329-3e52f47424c7 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:56.318666 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c55eba-66ce-4394-9329-3e52f47424c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:56.318944 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c55eba-66ce-4394-9329-3e52f47424c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:56.319670 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c55eba-66ce-4394-9329-3e52f47424c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:56.319977 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c55eba-66ce-4394-9329-3e52f47424c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:56.321370 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c55eba-66ce-4394-9329-3e52f47424c7 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:56.337943 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 545/1069] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:29:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:29:56.339224 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b49f3-54d0-40fd-9cf6-5daca1001f89 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:56.442086 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b49f3-54d0-40fd-9cf6-5daca1001f89 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:56.478583 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8109940-74b1-401d-8cfe-104c6ade44a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:56.479397 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8109940-74b1-401d-8cfe-104c6ade44a6 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:56.479684 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8109940-74b1-401d-8cfe-104c6ade44a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:56.479965 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8109940-74b1-401d-8cfe-104c6ade44a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:56.488108 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8109940-74b1-401d-8cfe-104c6ade44a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:56.489660 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8109940-74b1-401d-8cfe-104c6ade44a6 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:56.505781 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-d8109940-74b1-401d-8cfe-104c6ade44a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:29:56.505781 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:56.505781 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:56.505781 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:56.505781 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:56.505781 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:56.505781 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:56.505781 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:56.505781 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:56.505781 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:56.505781 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:56.505781 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:56.505781 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 16:29:56.505781 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 16:29:56.505781 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 16:29:56.506729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 16:29:56.506729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:56.506729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:56.506729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:56.506729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:56.506729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:56.506729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:56.506729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:56.506729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:56.506729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:56.506729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:56.506729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:56.506729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:56.506729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:56.506729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:56.506729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:56.506729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:56.507682 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 16:29:56.507682 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 16:29:56.507682 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 16:29:56.507682 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 16:29:56.507682 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 16:29:56.507682 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 16:29:56.507682 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:29:56.507682 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:56.508864 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 525/1070] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:29:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:56.512796 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c55eba-66ce-4394-9329-3e52f47424c7 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:56.618194 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c55eba-66ce-4394-9329-3e52f47424c7 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:56.652357 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:56.658276 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db68db92-cdbe-40fd-9979-f0edabb3634c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:56.659184 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68db92-cdbe-40fd-9979-f0edabb3634c None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:56.659668 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68db92-cdbe-40fd-9979-f0edabb3634c None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:56.660126 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68db92-cdbe-40fd-9979-f0edabb3634c None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:56.661112 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db68db92-cdbe-40fd-9979-f0edabb3634c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:56.661646 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db68db92-cdbe-40fd-9979-f0edabb3634c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:56.663280 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db68db92-cdbe-40fd-9979-f0edabb3634c None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:56.666431 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60eecae-119e-41bf-b207-834c920ccd96 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:56.666911 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60eecae-119e-41bf-b207-834c920ccd96 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:56.667000 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60eecae-119e-41bf-b207-834c920ccd96 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:56.668617 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 526/1071] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 16:29:56.673730 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f3268-8ed1-49a0-98cd-2aa8a88c1d42 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:56.674022 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f3268-8ed1-49a0-98cd-2aa8a88c1d42 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:56.674308 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f3268-8ed1-49a0-98cd-2aa8a88c1d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:56.676739 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 546/1072] 10.222.0.36 () {56 vars in 1148 bytes} [Thu May 19 16:29:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:56.682648 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db68db92-cdbe-40fd-9979-f0edabb3634c None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:56.692553 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-563f3268-8ed1-49a0-98cd-2aa8a88c1d42 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:56.693776 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-563f3268-8ed1-49a0-98cd-2aa8a88c1d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:56.726398 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:56.768828 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 527/1073] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 16:29:56.790643 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f3268-8ed1-49a0-98cd-2aa8a88c1d42 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:56.792990 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db68db92-cdbe-40fd-9979-f0edabb3634c None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:56.834793 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd10a2-b00f-42d9-9996-2eb9150f6d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:56.835816 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd10a2-b00f-42d9-9996-2eb9150f6d99 None admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:56.836103 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd10a2-b00f-42d9-9996-2eb9150f6d99 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:56.836385 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd10a2-b00f-42d9-9996-2eb9150f6d99 None admin] PATH_INFO: `/v3/projects/631c76794a124d6da22c626e590e4352/users/58bdeada00b74b7b8e186e6dc51e6dfb/roles/58a72d6ac3ff4e278e79059c4fe2ec5a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:56.839329 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bd10a2-b00f-42d9-9996-2eb9150f6d99 None admin] RBAC: Authorizing `identity:create_grant(project_id=631c76794a124d6da22c626e590e4352, user_id=58bdeada00b74b7b8e186e6dc51e6dfb, role_id=58a72d6ac3ff4e278e79059c4fe2ec5a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:56.840933 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bd10a2-b00f-42d9-9996-2eb9150f6d99 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:56.857193 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 547/1074] 10.222.0.36 () {58 vars in 1439 bytes} [Thu May 19 16:29:56 2022] PUT /identity/v3/projects/631c76794a124d6da22c626e590e4352/users/58bdeada00b74b7b8e186e6dc51e6dfb/roles/58a72d6ac3ff4e278e79059c4fe2ec5a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:29:56.914881 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f3268-8ed1-49a0-98cd-2aa8a88c1d42 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:56.954470 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5b1f6a-bf37-43d8-9fff-a5f52d537225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:56.955144 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b1f6a-bf37-43d8-9fff-a5f52d537225 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:56.955434 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b1f6a-bf37-43d8-9fff-a5f52d537225 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:56.955713 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b1f6a-bf37-43d8-9fff-a5f52d537225 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:56.992785 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:57.031045 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5b1f6a-bf37-43d8-9fff-a5f52d537225 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:57.032461 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5b1f6a-bf37-43d8-9fff-a5f52d537225 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:57.146039 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 528/1075] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:29:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 16:29:57.152736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd10a2-b00f-42d9-9996-2eb9150f6d99 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:57.220386 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:57.272062 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd10a2-b00f-42d9-9996-2eb9150f6d99 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:57.308871 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4a9677-759e-469d-8083-5b9454be4cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:57.309640 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a9677-759e-469d-8083-5b9454be4cf2 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:57.309929 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a9677-759e-469d-8083-5b9454be4cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:57.310210 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a9677-759e-469d-8083-5b9454be4cf2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:57.320827 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4a9677-759e-469d-8083-5b9454be4cf2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:57.322323 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4a9677-759e-469d-8083-5b9454be4cf2 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:57.340346 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-fd4a9677-759e-469d-8083-5b9454be4cf2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:57.340346 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:57.340346 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:57.340346 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:57.340346 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:57.340346 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:57.340346 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:57.340346 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:57.340346 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:57.340346 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:57.340346 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:57.340346 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:57.340346 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:29:57.340346 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:29:57.340346 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:29:57.341588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:29:57.341588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:57.341588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:57.341588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:57.341588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:57.341588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:57.341588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:57.341588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:57.341588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:57.341588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:57.341588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:57.341588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:57.341588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:57.341588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:57.341588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:57.341588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:57.341588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:57.343010 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:29:57.343010 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:29:57.343010 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:57.343010 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:57.343010 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 548/1076] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:29:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:57.348099 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5b1f6a-bf37-43d8-9fff-a5f52d537225 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:57.416713 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:57.451559 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5b1f6a-bf37-43d8-9fff-a5f52d537225 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:57.489327 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1cecec-a0c2-4d67-a360-f6eaa643a526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:57.490304 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1cecec-a0c2-4d67-a360-f6eaa643a526 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:57.490733 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1cecec-a0c2-4d67-a360-f6eaa643a526 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:57.491149 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1cecec-a0c2-4d67-a360-f6eaa643a526 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:57.492383 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1cecec-a0c2-4d67-a360-f6eaa643a526 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:57.492857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1cecec-a0c2-4d67-a360-f6eaa643a526 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:57.495030 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1cecec-a0c2-4d67-a360-f6eaa643a526 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:57.521182 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 529/1077] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:29:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:57.770553 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f28c74-ebe3-40a8-9a32-6d147dc61ea3 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:57.771082 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f28c74-ebe3-40a8-9a32-6d147dc61ea3 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:57.771177 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f28c74-ebe3-40a8-9a32-6d147dc61ea3 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:57.772497 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 549/1078] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:29:57.778097 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c1df8e-339c-41c5-b395-818b206160c6 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:57.778520 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c1df8e-339c-41c5-b395-818b206160c6 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:57.778694 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c1df8e-339c-41c5-b395-818b206160c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:57.796167 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-06c1df8e-339c-41c5-b395-818b206160c6 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:57.797403 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-06c1df8e-339c-41c5-b395-818b206160c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:57.868399 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 530/1079] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:29:57.891496 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f28c74-ebe3-40a8-9a32-6d147dc61ea3 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:57.991987 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f28c74-ebe3-40a8-9a32-6d147dc61ea3 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:57.994419 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:58.030461 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef84b82-ced1-4aa7-9791-115c1020e753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:58.031165 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef84b82-ced1-4aa7-9791-115c1020e753 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:58.031446 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef84b82-ced1-4aa7-9791-115c1020e753 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:58.031721 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef84b82-ced1-4aa7-9791-115c1020e753 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:58.041776 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef84b82-ced1-4aa7-9791-115c1020e753 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:29:58.049509 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-0ef84b82-ced1-4aa7-9791-115c1020e753 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:29:58.049509 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:58.049509 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:58.049509 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:58.049509 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:58.049509 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:58.049509 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:58.049509 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:58.049509 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:58.049509 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:58.049509 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:58.049509 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:58.049509 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:29:58.049509 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:29:58.049509 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:29:58.049509 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:29:58.050527 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:29:58.050527 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:29:58.050527 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:29:58.050527 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:29:58.050527 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:58.050527 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:58.050527 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:58.050527 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:58.050527 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:58.050527 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:58.050527 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:58.050527 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:29:58.050527 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:58.050527 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:58.050527 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:58.050527 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:58.051385 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:58.051385 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:58.051385 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:58.051385 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:58.051385 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:29:58.051385 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:29:58.051385 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:29:58.051385 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:29:58.051385 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:29:58.051385 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:29:58.051385 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:29:58.051385 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:29:58.051385 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 550/1080] 10.222.0.36 () {56 vars in 1096 bytes} [Thu May 19 16:29:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:58.056613 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c1df8e-339c-41c5-b395-818b206160c6 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:58.113787 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:58.156591 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c1df8e-339c-41c5-b395-818b206160c6 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:58.189555 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0805b667-5f6d-4e6f-babd-b566fae2ed5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:58.190222 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0805b667-5f6d-4e6f-babd-b566fae2ed5e None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:58.190510 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0805b667-5f6d-4e6f-babd-b566fae2ed5e None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:58.190787 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0805b667-5f6d-4e6f-babd-b566fae2ed5e None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:58.191532 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0805b667-5f6d-4e6f-babd-b566fae2ed5e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:58.191839 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0805b667-5f6d-4e6f-babd-b566fae2ed5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:58.193221 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0805b667-5f6d-4e6f-babd-b566fae2ed5e None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:58.201023 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 531/1081] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:29:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:58.207737 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef84b82-ced1-4aa7-9791-115c1020e753 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:58.303298 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef84b82-ced1-4aa7-9791-115c1020e753 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:58.336047 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e4f77e-a778-414b-843a-e2a9fcff9aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:58.336909 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4f77e-a778-414b-843a-e2a9fcff9aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:58.337221 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4f77e-a778-414b-843a-e2a9fcff9aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:58.337500 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4f77e-a778-414b-843a-e2a9fcff9aa3 None admin] PATH_INFO: `/v3/users/c52e6c529cd34730ae07a8e890f337a6` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:58.338667 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e4f77e-a778-414b-843a-e2a9fcff9aa3 None admin] RBAC: Authorizing `identity:get_user(user_id=c52e6c529cd34730ae07a8e890f337a6)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:58.340094 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e4f77e-a778-414b-843a-e2a9fcff9aa3 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:58.341737 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 551/1082] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:29:58 2022] GET /identity/v3/users/c52e6c529cd34730ae07a8e890f337a6 => generated 285 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:58.347439 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0805b667-5f6d-4e6f-babd-b566fae2ed5e None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:58.444918 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0805b667-5f6d-4e6f-babd-b566fae2ed5e None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:58.464127 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:58.478488 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf865ef-7d6e-468c-94bd-d2fd6069b350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:58.479362 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf865ef-7d6e-468c-94bd-d2fd6069b350 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:58.479649 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf865ef-7d6e-468c-94bd-d2fd6069b350 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:58.479927 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf865ef-7d6e-468c-94bd-d2fd6069b350 None admin] PATH_INFO: `/v3/projects/abba9b53b10946799633b36c9dee5f73` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:58.481107 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf865ef-7d6e-468c-94bd-d2fd6069b350 None admin] RBAC: Authorizing `identity:get_project(project_id=abba9b53b10946799633b36c9dee5f73)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:58.483214 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf865ef-7d6e-468c-94bd-d2fd6069b350 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:58.484925 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 532/1083] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:29:58 2022] GET /identity/v3/projects/abba9b53b10946799633b36c9dee5f73 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:29:58.490790 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e4f77e-a778-414b-843a-e2a9fcff9aa3 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:58.523089 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:58.592936 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e4f77e-a778-414b-843a-e2a9fcff9aa3 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:58.626617 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d2907-d894-41c2-aad5-1e331a5d1d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:58.627541 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d2907-d894-41c2-aad5-1e331a5d1d66 None admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:58.627809 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d2907-d894-41c2-aad5-1e331a5d1d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:58.628089 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d2907-d894-41c2-aad5-1e331a5d1d66 None admin] PATH_INFO: `/v3/projects/abba9b53b10946799633b36c9dee5f73/users/c52e6c529cd34730ae07a8e890f337a6/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:58.630093 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d2907-d894-41c2-aad5-1e331a5d1d66 None admin] RBAC: Authorizing `identity:create_grant(project_id=abba9b53b10946799633b36c9dee5f73, user_id=c52e6c529cd34730ae07a8e890f337a6, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:58.631548 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d2907-d894-41c2-aad5-1e331a5d1d66 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:58.646400 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 552/1084] 10.222.0.36 () {58 vars in 1439 bytes} [Thu May 19 16:29:58 2022] PUT /identity/v3/projects/abba9b53b10946799633b36c9dee5f73/users/c52e6c529cd34730ae07a8e890f337a6/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:29:59.118769 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66dda7b-62f5-447c-9a9e-bc3115d66a9c None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:59.119297 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66dda7b-62f5-447c-9a9e-bc3115d66a9c None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:59.119411 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66dda7b-62f5-447c-9a9e-bc3115d66a9c None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:59.120842 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 533/1085] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:29:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:29:59.126787 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146bbde-f10b-4631-9b75-ef6b15d85de1 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:59.127274 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146bbde-f10b-4631-9b75-ef6b15d85de1 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:59.127480 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146bbde-f10b-4631-9b75-ef6b15d85de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:59.145738 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-6146bbde-f10b-4631-9b75-ef6b15d85de1 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:29:59.146853 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-6146bbde-f10b-4631-9b75-ef6b15d85de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:29:59.216223 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 553/1086] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:29:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 16:29:59.239818 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66dda7b-62f5-447c-9a9e-bc3115d66a9c None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:59.339608 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66dda7b-62f5-447c-9a9e-bc3115d66a9c None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:59.373051 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16da5f8c-61b1-43bd-bf56-cf83bb053995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:59.374491 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16da5f8c-61b1-43bd-bf56-cf83bb053995 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:59.374776 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16da5f8c-61b1-43bd-bf56-cf83bb053995 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:59.375050 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16da5f8c-61b1-43bd-bf56-cf83bb053995 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:59.385329 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16da5f8c-61b1-43bd-bf56-cf83bb053995 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:29:59.393307 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-16da5f8c-61b1-43bd-bf56-cf83bb053995 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 16:29:59.393307 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:59.393307 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:59.393307 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:59.393307 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:59.393307 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:59.393307 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:59.393307 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:59.393307 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:59.393307 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:59.393307 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:59.393307 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:59.393307 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:29:59.393307 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:29:59.393307 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:29:59.393307 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:29:59.394689 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:29:59.394689 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:29:59.394689 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:29:59.394689 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:29:59.394689 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:59.394689 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:59.394689 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:59.394689 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:59.394689 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:59.394689 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:59.394689 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:59.394689 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:59.394689 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:59.394689 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:59.394689 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:59.394689 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:59.395690 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:59.395690 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:59.395690 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:59.395690 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:59.395690 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:29:59.395690 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:29:59.395690 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:29:59.395690 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:29:59.395690 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:29:59.395690 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:29:59.395690 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 16:29:59.395690 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:59.395690 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 534/1087] 10.222.0.36 () {56 vars in 1120 bytes} [Thu May 19 16:29:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:59.400950 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6146bbde-f10b-4631-9b75-ef6b15d85de1 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:59.424722 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:59.500956 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6146bbde-f10b-4631-9b75-ef6b15d85de1 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:59.535773 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024a6502-fbcd-42cd-8320-5b211e812e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:59.536435 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a6502-fbcd-42cd-8320-5b211e812e01 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:59.536721 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a6502-fbcd-42cd-8320-5b211e812e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:59.536997 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a6502-fbcd-42cd-8320-5b211e812e01 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:59.537786 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024a6502-fbcd-42cd-8320-5b211e812e01 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:59.538098 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024a6502-fbcd-42cd-8320-5b211e812e01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:59.539495 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024a6502-fbcd-42cd-8320-5b211e812e01 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:59.548216 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 554/1088] 10.222.0.36 () {56 vars in 1115 bytes} [Thu May 19 16:29:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:59.557843 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16da5f8c-61b1-43bd-bf56-cf83bb053995 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:59.608665 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:59.660707 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16da5f8c-61b1-43bd-bf56-cf83bb053995 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:59.693325 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae6300b-702e-43a9-b528-f6ada24982ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:59.694122 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae6300b-702e-43a9-b528-f6ada24982ac None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:59.694411 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae6300b-702e-43a9-b528-f6ada24982ac None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:59.694692 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae6300b-702e-43a9-b528-f6ada24982ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:59.705768 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae6300b-702e-43a9-b528-f6ada24982ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:59.707216 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae6300b-702e-43a9-b528-f6ada24982ac None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:59.723894 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-3ae6300b-702e-43a9-b528-f6ada24982ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:59.723894 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:29:59.723894 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:29:59.723894 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:29:59.723894 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:29:59.723894 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:29:59.723894 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:29:59.723894 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:29:59.723894 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:29:59.723894 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:29:59.723894 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:29:59.723894 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:29:59.723894 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:29:59.723894 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:29:59.723894 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:29:59.724985 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:29:59.724985 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:29:59.724985 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:29:59.724985 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:29:59.724985 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:29:59.724985 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:29:59.724985 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:29:59.724985 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:29:59.724985 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:29:59.724985 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:29:59.724985 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:29:59.724985 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:29:59.724985 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:29:59.724985 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:29:59.724985 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:29:59.724985 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:29:59.724985 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:29:59.725951 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:29:59.725951 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:29:59.725951 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:29:59.725951 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:29:59.725951 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 535/1089] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:29:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:29:59.731126 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024a6502-fbcd-42cd-8320-5b211e812e01 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:59.820362 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:29:59.836486 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024a6502-fbcd-42cd-8320-5b211e812e01 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:29:59.870239 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5083a261-ff21-49c3-a9a6-92245711ddec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:29:59.870908 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5083a261-ff21-49c3-a9a6-92245711ddec None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:29:59.871202 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5083a261-ff21-49c3-a9a6-92245711ddec None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:29:59.871481 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5083a261-ff21-49c3-a9a6-92245711ddec None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:29:59.872212 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5083a261-ff21-49c3-a9a6-92245711ddec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:29:59.872528 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5083a261-ff21-49c3-a9a6-92245711ddec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:29:59.873939 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5083a261-ff21-49c3-a9a6-92245711ddec None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:29:59.890679 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 555/1090] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:29:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:29:59.898050 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae6300b-702e-43a9-b528-f6ada24982ac None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:29:59.997348 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:00.000255 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae6300b-702e-43a9-b528-f6ada24982ac None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:00.033839 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772e3cbc-f212-44b5-8080-d9c7ca5035d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:00.034627 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e3cbc-f212-44b5-8080-d9c7ca5035d3 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:00.034916 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e3cbc-f212-44b5-8080-d9c7ca5035d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:00.035198 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e3cbc-f212-44b5-8080-d9c7ca5035d3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:00.042655 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772e3cbc-f212-44b5-8080-d9c7ca5035d3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:00.044094 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772e3cbc-f212-44b5-8080-d9c7ca5035d3 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:00.057898 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-772e3cbc-f212-44b5-8080-d9c7ca5035d3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 16:30:00.057898 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:30:00.057898 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:30:00.057898 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:30:00.057898 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:30:00.057898 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:30:00.057898 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:30:00.057898 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:30:00.057898 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:30:00.057898 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:30:00.057898 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:30:00.057898 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:30:00.057898 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 16:30:00.057898 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 16:30:00.057898 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 16:30:00.057898 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 16:30:00.058962 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:30:00.058962 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:30:00.058962 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 16:30:00.058962 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:30:00.058962 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 16:30:00.058962 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:30:00.058962 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:30:00.058962 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:30:00.058962 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:30:00.058962 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:30:00.058962 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:30:00.058962 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:30:00.058962 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:30:00.058962 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:30:00.058962 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:30:00.058962 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:30:00.058962 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:30:00.058962 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:30:00.059927 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:30:00.059927 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:30:00.059927 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 16:30:00.059927 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 16:30:00.059927 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 16:30:00.059927 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 16:30:00.059927 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 16:30:00.059927 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 16:30:00.059927 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 16:30:00.059927 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:30:00.059927 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 536/1091] 10.222.0.36 () {56 vars in 1093 bytes} [Thu May 19 16:29:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:30:00.065055 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5083a261-ff21-49c3-a9a6-92245711ddec None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:00.162429 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5083a261-ff21-49c3-a9a6-92245711ddec None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:00.197017 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a2337f-c7f9-46fc-92e9-e2be776475c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:00.197715 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2337f-c7f9-46fc-92e9-e2be776475c2 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:00.198001 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2337f-c7f9-46fc-92e9-e2be776475c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:00.198280 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2337f-c7f9-46fc-92e9-e2be776475c2 None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:00.199126 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a2337f-c7f9-46fc-92e9-e2be776475c2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:30:00.199443 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a2337f-c7f9-46fc-92e9-e2be776475c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:00.200886 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a2337f-c7f9-46fc-92e9-e2be776475c2 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:00.223268 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 556/1092] 10.222.0.36 () {56 vars in 1133 bytes} [Thu May 19 16:30:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:30:00.224446 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772e3cbc-f212-44b5-8080-d9c7ca5035d3 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:00.321698 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772e3cbc-f212-44b5-8080-d9c7ca5035d3 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:00.354260 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f70f7-6c5e-4322-ab79-be4d369c4174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:00.354913 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f70f7-6c5e-4322-ab79-be4d369c4174 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:00.355201 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f70f7-6c5e-4322-ab79-be4d369c4174 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:00.355482 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f70f7-6c5e-4322-ab79-be4d369c4174 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:00.383159 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:00.422899 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04f70f7-6c5e-4322-ab79-be4d369c4174 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:00.424342 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04f70f7-6c5e-4322-ab79-be4d369c4174 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:00.525621 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 537/1093] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:30:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 16:30:00.526849 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a2337f-c7f9-46fc-92e9-e2be776475c2 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:00.529753 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:00.624867 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a2337f-c7f9-46fc-92e9-e2be776475c2 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:00.661247 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde5969-6cc3-43d9-ba97-b5f466bfc540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:00.662064 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde5969-6cc3-43d9-ba97-b5f466bfc540 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:00.662352 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde5969-6cc3-43d9-ba97-b5f466bfc540 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:00.662632 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde5969-6cc3-43d9-ba97-b5f466bfc540 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:00.674860 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efde5969-6cc3-43d9-ba97-b5f466bfc540 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:00.676361 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efde5969-6cc3-43d9-ba97-b5f466bfc540 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:00.695965 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-efde5969-6cc3-43d9-ba97-b5f466bfc540 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:30:00.695965 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:30:00.695965 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:30:00.695965 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:30:00.695965 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:30:00.695965 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:30:00.695965 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:30:00.695965 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:30:00.695965 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:30:00.695965 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:30:00.695965 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:30:00.695965 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:30:00.695965 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:30:00.695965 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:30:00.695965 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:30:00.697112 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:30:00.697112 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:30:00.697112 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:30:00.697112 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:30:00.697112 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:30:00.697112 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:30:00.697112 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:30:00.697112 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:30:00.697112 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:30:00.697112 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:30:00.697112 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:30:00.697112 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:30:00.697112 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:30:00.697112 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:30:00.697112 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:30:00.697112 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:30:00.697112 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:30:00.698104 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:30:00.698104 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:30:00.698104 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:30:00.698104 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:30:00.698104 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 557/1094] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:30:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 16:30:00.703224 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f70f7-6c5e-4322-ab79-be4d369c4174 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:00.760606 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:00.801238 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f70f7-6c5e-4322-ab79-be4d369c4174 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:00.835643 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3871c3-7033-442b-baf9-424f6889e391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:00.836316 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3871c3-7033-442b-baf9-424f6889e391 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:00.836602 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3871c3-7033-442b-baf9-424f6889e391 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:00.836880 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3871c3-7033-442b-baf9-424f6889e391 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:00.837658 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3871c3-7033-442b-baf9-424f6889e391 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:30:00.837967 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3871c3-7033-442b-baf9-424f6889e391 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:00.839349 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3871c3-7033-442b-baf9-424f6889e391 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:00.851649 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 538/1095] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:30:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:30:00.857267 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab23c5-f809-445f-8e8e-70293a430aac None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:00.857944 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3871c3-7033-442b-baf9-424f6889e391 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:00.858104 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab23c5-f809-445f-8e8e-70293a430aac None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:00.858387 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab23c5-f809-445f-8e8e-70293a430aac None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:00.859752 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 558/1096] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:30:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:30:00.864842 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc61645-126b-4c24-ac70-86090944fddb None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:00.865226 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc61645-126b-4c24-ac70-86090944fddb None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:00.865431 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc61645-126b-4c24-ac70-86090944fddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:00.882286 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-5cc61645-126b-4c24-ac70-86090944fddb None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:30:00.883394 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-5cc61645-126b-4c24-ac70-86090944fddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:30:00.934008 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:00.953060 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 559/1097] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:30:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 16:30:00.974784 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3871c3-7033-442b-baf9-424f6889e391 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:00.979421 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc61645-126b-4c24-ac70-86090944fddb None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:01.010253 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:01.036267 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f70b93-0f5d-4df6-9b12-e57f4057b334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:01.037074 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f70b93-0f5d-4df6-9b12-e57f4057b334 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:01.037389 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f70b93-0f5d-4df6-9b12-e57f4057b334 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:01.037663 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f70b93-0f5d-4df6-9b12-e57f4057b334 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:01.047442 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f70b93-0f5d-4df6-9b12-e57f4057b334 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:01.048899 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f70b93-0f5d-4df6-9b12-e57f4057b334 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:01.069751 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-e7f70b93-0f5d-4df6-9b12-e57f4057b334 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:30:01.069751 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:30:01.069751 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:30:01.069751 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:30:01.069751 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:30:01.069751 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:30:01.069751 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:30:01.069751 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:30:01.069751 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:30:01.069751 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:30:01.069751 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:30:01.069751 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:30:01.069751 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 16:30:01.069751 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 16:30:01.069751 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 16:30:01.070909 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 16:30:01.070909 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:30:01.070909 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:30:01.070909 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:30:01.070909 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:30:01.070909 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:30:01.070909 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:30:01.070909 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:30:01.070909 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:30:01.070909 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:30:01.070909 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:30:01.070909 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:30:01.070909 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:30:01.070909 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:30:01.070909 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:30:01.070909 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:30:01.070909 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:30:01.074445 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 16:30:01.074445 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 16:30:01.074445 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 16:30:01.074445 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 16:30:01.074445 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 16:30:01.074445 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 16:30:01.074445 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:30:01.074445 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:30:01.074445 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 539/1098] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:30:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 16:30:01.077322 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f70b93-0f5d-4df6-9b12-e57f4057b334 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:01.122076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc61645-126b-4c24-ac70-86090944fddb None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:01.163412 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3146d06b-9fc5-41b7-b201-869df08ac71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:01.164148 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146d06b-9fc5-41b7-b201-869df08ac71b None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:01.164473 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146d06b-9fc5-41b7-b201-869df08ac71b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:01.164701 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146d06b-9fc5-41b7-b201-869df08ac71b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:01.177826 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3146d06b-9fc5-41b7-b201-869df08ac71b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:30:01.186467 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-3146d06b-9fc5-41b7-b201-869df08ac71b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:30:01.186467 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:30:01.186467 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:30:01.186467 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:30:01.186467 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:30:01.186467 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:30:01.186467 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:30:01.186467 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:30:01.186467 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:30:01.186467 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:30:01.186467 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:30:01.186467 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:30:01.186467 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:30:01.186467 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:30:01.186467 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:30:01.186467 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:30:01.187502 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:30:01.187502 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:30:01.187502 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:30:01.187502 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:30:01.187502 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:30:01.187502 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:30:01.187502 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:30:01.187502 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:30:01.187502 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:30:01.187502 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:30:01.187502 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:30:01.187502 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:30:01.187502 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:30:01.187502 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:30:01.187502 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:30:01.187502 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:30:01.188344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:30:01.188344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:30:01.188344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:30:01.188344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:30:01.188344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:30:01.188344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:30:01.188344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:30:01.188344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:30:01.188344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:30:01.188344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:30:01.188344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:30:01.188344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:30:01.190414 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 560/1099] 10.222.0.36 () {56 vars in 1096 bytes} [Thu May 19 16:30:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:30:01.193852 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3146d06b-9fc5-41b7-b201-869df08ac71b None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:01.212879 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f70b93-0f5d-4df6-9b12-e57f4057b334 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:01.250270 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:01.260612 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9e789d-1c29-4fdc-b243-bc70d7acf91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:01.261422 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e789d-1c29-4fdc-b243-bc70d7acf91f None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:01.261671 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e789d-1c29-4fdc-b243-bc70d7acf91f None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:01.261899 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e789d-1c29-4fdc-b243-bc70d7acf91f None admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:01.262704 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9e789d-1c29-4fdc-b243-bc70d7acf91f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:30:01.263362 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9e789d-1c29-4fdc-b243-bc70d7acf91f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:01.265696 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9e789d-1c29-4fdc-b243-bc70d7acf91f None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:01.286619 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 540/1100] 10.222.0.36 () {56 vars in 1148 bytes} [Thu May 19 16:30:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:30:01.293529 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9e789d-1c29-4fdc-b243-bc70d7acf91f None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:01.324108 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3146d06b-9fc5-41b7-b201-869df08ac71b None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:01.342183 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:01.372840 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea9878f-d61f-48e5-8952-e4cee8feaf80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:01.374328 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9878f-d61f-48e5-8952-e4cee8feaf80 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:01.374617 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9878f-d61f-48e5-8952-e4cee8feaf80 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:01.375170 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9878f-d61f-48e5-8952-e4cee8feaf80 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:01.376233 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea9878f-d61f-48e5-8952-e4cee8feaf80 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:30:01.376554 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea9878f-d61f-48e5-8952-e4cee8feaf80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:01.378350 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea9878f-d61f-48e5-8952-e4cee8feaf80 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:01.387886 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 561/1101] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:30:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:30:01.394584 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea9878f-d61f-48e5-8952-e4cee8feaf80 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:01.418306 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9e789d-1c29-4fdc-b243-bc70d7acf91f None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:01.453321 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12ee232-0e3b-434d-91c0-aafea1fe775d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:01.453977 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ee232-0e3b-434d-91c0-aafea1fe775d None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:01.454269 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ee232-0e3b-434d-91c0-aafea1fe775d None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:01.454621 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ee232-0e3b-434d-91c0-aafea1fe775d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:01.455365 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12ee232-0e3b-434d-91c0-aafea1fe775d None admin] RBAC: Adding query filter params (role.id=58a72d6ac3ff4e278e79059c4fe2ec5a, scope.project.id=631c76794a124d6da22c626e590e4352, user.id=58bdeada00b74b7b8e186e6dc51e6dfb) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:30:01.455676 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12ee232-0e3b-434d-91c0-aafea1fe775d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:01.457110 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12ee232-0e3b-434d-91c0-aafea1fe775d None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:01.463006 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 541/1102] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:30:01 2022] GET /identity/v3/role_assignments?user.id=58bdeada00b74b7b8e186e6dc51e6dfb&scope.project.id=631c76794a124d6da22c626e590e4352&role.id=58a72d6ac3ff4e278e79059c4fe2ec5a => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:30:01.496721 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea9878f-d61f-48e5-8952-e4cee8feaf80 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:01.505246 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:01.535056 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f826119f-f0bc-4514-898b-1755c93c93dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:01.535951 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f826119f-f0bc-4514-898b-1755c93c93dd None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:01.536233 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f826119f-f0bc-4514-898b-1755c93c93dd None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:01.536505 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f826119f-f0bc-4514-898b-1755c93c93dd None admin] PATH_INFO: `/v3/users/c52e6c529cd34730ae07a8e890f337a6` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:01.537776 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f826119f-f0bc-4514-898b-1755c93c93dd None admin] RBAC: Authorizing `identity:get_user(user_id=c52e6c529cd34730ae07a8e890f337a6)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:01.539240 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f826119f-f0bc-4514-898b-1755c93c93dd None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:01.540928 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 562/1103] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:30:01 2022] GET /identity/v3/users/c52e6c529cd34730ae07a8e890f337a6 => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:30:01.547978 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12ee232-0e3b-434d-91c0-aafea1fe775d None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:01.655841 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12ee232-0e3b-434d-91c0-aafea1fe775d None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:01.658278 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:01.692775 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcff320-f511-45d9-9e33-a64c9aeb66df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:01.693383 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcff320-f511-45d9-9e33-a64c9aeb66df None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:01.693672 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcff320-f511-45d9-9e33-a64c9aeb66df None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:01.693953 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcff320-f511-45d9-9e33-a64c9aeb66df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:01.761773 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcff320-f511-45d9-9e33-a64c9aeb66df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:01.763241 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcff320-f511-45d9-9e33-a64c9aeb66df None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:01.864128 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 542/1104] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:30:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 16:30:01.870899 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f826119f-f0bc-4514-898b-1755c93c93dd None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:01.982015 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f826119f-f0bc-4514-898b-1755c93c93dd None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:02.016988 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64779d4a-351b-408b-b2bb-f961a42df703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:02.019866 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64779d4a-351b-408b-b2bb-f961a42df703 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:02.019866 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64779d4a-351b-408b-b2bb-f961a42df703 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:02.019866 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64779d4a-351b-408b-b2bb-f961a42df703 None admin] PATH_INFO: `/v3/projects/abba9b53b10946799633b36c9dee5f73` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:02.019866 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64779d4a-351b-408b-b2bb-f961a42df703 None admin] RBAC: Authorizing `identity:get_project(project_id=abba9b53b10946799633b36c9dee5f73)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:02.021371 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64779d4a-351b-408b-b2bb-f961a42df703 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:02.023077 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 563/1105] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:30:01 2022] GET /identity/v3/projects/abba9b53b10946799633b36c9dee5f73 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:30:02.029912 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcff320-f511-45d9-9e33-a64c9aeb66df None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:02.033188 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:02.129287 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcff320-f511-45d9-9e33-a64c9aeb66df None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:02.167781 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e55d83-86b9-4270-ab06-bda2d8257f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:02.168443 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e55d83-86b9-4270-ab06-bda2d8257f82 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:02.168732 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e55d83-86b9-4270-ab06-bda2d8257f82 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:02.169013 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e55d83-86b9-4270-ab06-bda2d8257f82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:02.169930 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e55d83-86b9-4270-ab06-bda2d8257f82 None admin] RBAC: Adding query filter params (role.id=e2915fe109244a07a9be96bd2b219fdd, scope.project.id=abba9b53b10946799633b36c9dee5f73, user.id=c52e6c529cd34730ae07a8e890f337a6) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:30:02.170246 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e55d83-86b9-4270-ab06-bda2d8257f82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:02.171686 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e55d83-86b9-4270-ab06-bda2d8257f82 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:02.177862 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 543/1106] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:30:02 2022] GET /identity/v3/role_assignments?user.id=c52e6c529cd34730ae07a8e890f337a6&scope.project.id=abba9b53b10946799633b36c9dee5f73&role.id=e2915fe109244a07a9be96bd2b219fdd => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:30:04.422828 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1a6d8-252e-4e3c-854d-5b5f0abbd577 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:04.423361 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1a6d8-252e-4e3c-854d-5b5f0abbd577 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:04.423457 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1a6d8-252e-4e3c-854d-5b5f0abbd577 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:04.424784 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 564/1107] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:30:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:30:04.430067 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697880d5-8e0e-4022-ae35-ea55e88f4cbb None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:04.430462 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697880d5-8e0e-4022-ae35-ea55e88f4cbb None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:04.430631 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697880d5-8e0e-4022-ae35-ea55e88f4cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:04.447516 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-697880d5-8e0e-4022-ae35-ea55e88f4cbb None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:30:04.448693 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-697880d5-8e0e-4022-ae35-ea55e88f4cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:30:04.507796 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:04.513282 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 544/1108] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:30:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:30:04.538292 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e1a6d8-252e-4e3c-854d-5b5f0abbd577 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:04.582810 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:04.641252 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e1a6d8-252e-4e3c-854d-5b5f0abbd577 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:04.673849 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4795b23-5c6c-45ec-9824-a9c6410b8edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:04.674579 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4795b23-5c6c-45ec-9824-a9c6410b8edf None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:04.674867 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4795b23-5c6c-45ec-9824-a9c6410b8edf None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:04.675147 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4795b23-5c6c-45ec-9824-a9c6410b8edf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:04.676316 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4795b23-5c6c-45ec-9824-a9c6410b8edf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:04.677796 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4795b23-5c6c-45ec-9824-a9c6410b8edf None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:04.679429 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 565/1109] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:30:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:30:04.685963 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697880d5-8e0e-4022-ae35-ea55e88f4cbb None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:04.782081 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697880d5-8e0e-4022-ae35-ea55e88f4cbb None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:04.817494 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0383ca4c-b30c-450a-89d0-2ef2f350e32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:04.818164 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383ca4c-b30c-450a-89d0-2ef2f350e32a None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:04.818449 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383ca4c-b30c-450a-89d0-2ef2f350e32a None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:04.818727 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383ca4c-b30c-450a-89d0-2ef2f350e32a None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:04.819435 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0383ca4c-b30c-450a-89d0-2ef2f350e32a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:04.820870 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0383ca4c-b30c-450a-89d0-2ef2f350e32a None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:04.840369 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 545/1110] 10.222.0.36 () {60 vars in 1132 bytes} [Thu May 19 16:30:04 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:30:07.144220 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac633c3c-c17f-4ac6-aba9-5b2d73ae5d27 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:07.144730 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac633c3c-c17f-4ac6-aba9-5b2d73ae5d27 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:07.144730 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac633c3c-c17f-4ac6-aba9-5b2d73ae5d27 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:07.146088 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 566/1111] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:30:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:30:07.151337 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec732c-3b87-468c-a87e-1ad8c8b325f3 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:07.151775 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec732c-3b87-468c-a87e-1ad8c8b325f3 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:07.151901 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec732c-3b87-468c-a87e-1ad8c8b325f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:07.168680 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-56ec732c-3b87-468c-a87e-1ad8c8b325f3 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:30:07.170449 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-56ec732c-3b87-468c-a87e-1ad8c8b325f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:30:07.217551 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:07.235677 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 546/1112] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:30:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:30:07.258839 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac633c3c-c17f-4ac6-aba9-5b2d73ae5d27 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:07.360795 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac633c3c-c17f-4ac6-aba9-5b2d73ae5d27 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:07.371095 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:07.395854 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b130e6-e81d-4304-b10d-3e1514f1528d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:07.396652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b130e6-e81d-4304-b10d-3e1514f1528d None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:07.396938 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b130e6-e81d-4304-b10d-3e1514f1528d None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:07.397255 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b130e6-e81d-4304-b10d-3e1514f1528d None admin] PATH_INFO: `/v3/roles/50f62a33eb434682baa7529619fe14b0` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:07.398745 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b130e6-e81d-4304-b10d-3e1514f1528d None admin] RBAC: Authorizing `identity:get_role(role_id=50f62a33eb434682baa7529619fe14b0)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:07.400127 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b130e6-e81d-4304-b10d-3e1514f1528d None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:07.401551 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 567/1113] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:30:07 2022] GET /identity/v3/roles/50f62a33eb434682baa7529619fe14b0 => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:30:07.408035 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec732c-3b87-468c-a87e-1ad8c8b325f3 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:07.502546 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec732c-3b87-468c-a87e-1ad8c8b325f3 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:07.535876 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b860d9-1649-4a91-8681-1a4b0a818389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:07.536737 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b860d9-1649-4a91-8681-1a4b0a818389 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:07.537021 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b860d9-1649-4a91-8681-1a4b0a818389 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:07.537333 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b860d9-1649-4a91-8681-1a4b0a818389 None admin] PATH_INFO: `/v3/users/7aebda29924f48c882ffd12fc4448d17` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:07.550398 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b860d9-1649-4a91-8681-1a4b0a818389 None admin] RBAC: Authorizing `identity:get_user(user_id=7aebda29924f48c882ffd12fc4448d17)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:07.552032 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b860d9-1649-4a91-8681-1a4b0a818389 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:07.553707 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 547/1114] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:30:07 2022] GET /identity/v3/users/7aebda29924f48c882ffd12fc4448d17 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:30:07.559486 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b130e6-e81d-4304-b10d-3e1514f1528d None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:07.661548 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b130e6-e81d-4304-b10d-3e1514f1528d None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:07.693730 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf70c96-d5bc-416b-a088-66f0e29cb6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:07.694673 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf70c96-d5bc-416b-a088-66f0e29cb6c8 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:07.694673 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf70c96-d5bc-416b-a088-66f0e29cb6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:07.694923 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf70c96-d5bc-416b-a088-66f0e29cb6c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:07.739782 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:07.761844 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf70c96-d5bc-416b-a088-66f0e29cb6c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:07.763412 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf70c96-d5bc-416b-a088-66f0e29cb6c8 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:07.863285 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 568/1115] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:30:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 16:30:07.869437 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b860d9-1649-4a91-8681-1a4b0a818389 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:07.921758 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:07.980320 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b860d9-1649-4a91-8681-1a4b0a818389 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:08.012970 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4803df7-d8f5-4a22-973b-f1af58dcd263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:08.013889 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4803df7-d8f5-4a22-973b-f1af58dcd263 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:08.014179 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4803df7-d8f5-4a22-973b-f1af58dcd263 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:08.014461 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4803df7-d8f5-4a22-973b-f1af58dcd263 None admin] PATH_INFO: `/v3/projects/abba9b53b10946799633b36c9dee5f73` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:08.015637 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4803df7-d8f5-4a22-973b-f1af58dcd263 None admin] RBAC: Authorizing `identity:get_project(project_id=abba9b53b10946799633b36c9dee5f73)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:08.017092 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4803df7-d8f5-4a22-973b-f1af58dcd263 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:08.019057 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 548/1116] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:30:07 2022] GET /identity/v3/projects/abba9b53b10946799633b36c9dee5f73 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:30:08.025384 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf70c96-d5bc-416b-a088-66f0e29cb6c8 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:08.101800 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:08.123923 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf70c96-d5bc-416b-a088-66f0e29cb6c8 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:08.159398 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e20422f-cd9f-4ba1-be09-8d1fd462c643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:08.160064 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20422f-cd9f-4ba1-be09-8d1fd462c643 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:08.160354 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20422f-cd9f-4ba1-be09-8d1fd462c643 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:08.160635 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20422f-cd9f-4ba1-be09-8d1fd462c643 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:08.161516 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e20422f-cd9f-4ba1-be09-8d1fd462c643 None admin] RBAC: Adding query filter params (role.id=50f62a33eb434682baa7529619fe14b0, scope.project.id=abba9b53b10946799633b36c9dee5f73, user.id=7aebda29924f48c882ffd12fc4448d17) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:30:08.161836 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e20422f-cd9f-4ba1-be09-8d1fd462c643 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:08.163286 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e20422f-cd9f-4ba1-be09-8d1fd462c643 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:08.168779 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 569/1117] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:30:08 2022] GET /identity/v3/role_assignments?user.id=7aebda29924f48c882ffd12fc4448d17&scope.project.id=abba9b53b10946799633b36c9dee5f73&role.id=50f62a33eb434682baa7529619fe14b0 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:30:10.388630 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dc471c-a83a-43a9-bd8e-39c1ad0f5acc None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:10.389195 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dc471c-a83a-43a9-bd8e-39c1ad0f5acc None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:10.389195 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dc471c-a83a-43a9-bd8e-39c1ad0f5acc None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:10.391247 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 549/1118] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:30:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:30:10.396686 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29d89e-058b-4c2c-a788-0d4529bedf93 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:10.397104 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29d89e-058b-4c2c-a788-0d4529bedf93 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:10.397319 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29d89e-058b-4c2c-a788-0d4529bedf93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:10.413995 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-8c29d89e-058b-4c2c-a788-0d4529bedf93 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:30:10.415218 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-8c29d89e-058b-4c2c-a788-0d4529bedf93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:30:10.480962 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 570/1119] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:30:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:30:10.504585 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dc471c-a83a-43a9-bd8e-39c1ad0f5acc None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:10.601941 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dc471c-a83a-43a9-bd8e-39c1ad0f5acc None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:10.613242 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:10.649949 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0e20c-0a82-4741-9e8d-e9ace17205b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:10.650799 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0e20c-0a82-4741-9e8d-e9ace17205b7 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:10.651086 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0e20c-0a82-4741-9e8d-e9ace17205b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:10.651367 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0e20c-0a82-4741-9e8d-e9ace17205b7 None admin] PATH_INFO: `/v3/roles/50f62a33eb434682baa7529619fe14b0` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:10.653248 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac0e20c-0a82-4741-9e8d-e9ace17205b7 None admin] RBAC: Authorizing `identity:get_role(role_id=50f62a33eb434682baa7529619fe14b0)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:10.654639 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac0e20c-0a82-4741-9e8d-e9ace17205b7 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:10.656070 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 550/1120] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:30:10 2022] GET /identity/v3/roles/50f62a33eb434682baa7529619fe14b0 => generated 223 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:30:10.662170 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c29d89e-058b-4c2c-a788-0d4529bedf93 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:10.749033 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:10.778084 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c29d89e-058b-4c2c-a788-0d4529bedf93 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:10.811762 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5953a254-a31d-493d-a9bd-ade875057daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:10.812631 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953a254-a31d-493d-a9bd-ade875057daa None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:10.812912 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953a254-a31d-493d-a9bd-ade875057daa None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:10.813210 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953a254-a31d-493d-a9bd-ade875057daa None admin] PATH_INFO: `/v3/users/7aebda29924f48c882ffd12fc4448d17` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:10.814379 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5953a254-a31d-493d-a9bd-ade875057daa None admin] RBAC: Authorizing `identity:get_user(user_id=7aebda29924f48c882ffd12fc4448d17)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:10.815825 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5953a254-a31d-493d-a9bd-ade875057daa None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:10.817479 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 571/1121] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:30:10 2022] GET /identity/v3/users/7aebda29924f48c882ffd12fc4448d17 => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:30:10.823217 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0e20c-0a82-4741-9e8d-e9ace17205b7 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:10.924062 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0e20c-0a82-4741-9e8d-e9ace17205b7 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:10.956570 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5332a9-c763-4f9d-b38d-3eb944e68011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:10.957464 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5332a9-c763-4f9d-b38d-3eb944e68011 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:10.957750 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5332a9-c763-4f9d-b38d-3eb944e68011 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:10.958031 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5332a9-c763-4f9d-b38d-3eb944e68011 None admin] PATH_INFO: `/v3/projects/abba9b53b10946799633b36c9dee5f73` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:10.959201 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5332a9-c763-4f9d-b38d-3eb944e68011 None admin] RBAC: Authorizing `identity:get_project(project_id=abba9b53b10946799633b36c9dee5f73)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:10.960632 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5332a9-c763-4f9d-b38d-3eb944e68011 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:10.962345 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 551/1122] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:30:10 2022] GET /identity/v3/projects/abba9b53b10946799633b36c9dee5f73 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:30:10.968856 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5953a254-a31d-493d-a9bd-ade875057daa None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:11.081084 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5953a254-a31d-493d-a9bd-ade875057daa None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:11.115867 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1772a77d-0387-4690-844d-0bffaa149016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:11.116742 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1772a77d-0387-4690-844d-0bffaa149016 None admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:11.117032 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1772a77d-0387-4690-844d-0bffaa149016 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:11.117337 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1772a77d-0387-4690-844d-0bffaa149016 None admin] PATH_INFO: `/v3/projects/abba9b53b10946799633b36c9dee5f73/users/7aebda29924f48c882ffd12fc4448d17/roles/50f62a33eb434682baa7529619fe14b0` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:11.119617 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1772a77d-0387-4690-844d-0bffaa149016 None admin] RBAC: Authorizing `identity:create_grant(project_id=abba9b53b10946799633b36c9dee5f73, user_id=7aebda29924f48c882ffd12fc4448d17, role_id=50f62a33eb434682baa7529619fe14b0)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:11.121119 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1772a77d-0387-4690-844d-0bffaa149016 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:11.131862 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:11.137405 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 572/1123] 10.222.0.36 () {58 vars in 1439 bytes} [Thu May 19 16:30:10 2022] PUT /identity/v3/projects/abba9b53b10946799633b36c9dee5f73/users/7aebda29924f48c882ffd12fc4448d17/roles/50f62a33eb434682baa7529619fe14b0 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:30:13.431110 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc5025-d51f-4c1f-9d31-501a5c9a63b2 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:13.431643 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc5025-d51f-4c1f-9d31-501a5c9a63b2 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:13.431753 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc5025-d51f-4c1f-9d31-501a5c9a63b2 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:13.433051 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 552/1124] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:30:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:30:13.438861 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a972c31c-c101-48e4-a576-f3492beea012 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:13.439303 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a972c31c-c101-48e4-a576-f3492beea012 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:13.439396 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a972c31c-c101-48e4-a576-f3492beea012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:13.455859 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-a972c31c-c101-48e4-a576-f3492beea012 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:30:13.457047 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-a972c31c-c101-48e4-a576-f3492beea012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:30:13.522132 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 573/1125] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:30:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:30:13.545267 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc5025-d51f-4c1f-9d31-501a5c9a63b2 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:13.593418 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:13.645720 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc5025-d51f-4c1f-9d31-501a5c9a63b2 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:13.678563 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb98e81-d6f4-45aa-b627-4a1defd9daff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:13.679418 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb98e81-d6f4-45aa-b627-4a1defd9daff None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:13.679702 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb98e81-d6f4-45aa-b627-4a1defd9daff None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:13.679978 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb98e81-d6f4-45aa-b627-4a1defd9daff None admin] PATH_INFO: `/v3/roles/50f62a33eb434682baa7529619fe14b0` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:13.681529 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb98e81-d6f4-45aa-b627-4a1defd9daff None admin] RBAC: Authorizing `identity:get_role(role_id=50f62a33eb434682baa7529619fe14b0)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:13.682841 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb98e81-d6f4-45aa-b627-4a1defd9daff None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:13.684208 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 553/1126] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:30:13 2022] GET /identity/v3/roles/50f62a33eb434682baa7529619fe14b0 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:30:13.690512 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a972c31c-c101-48e4-a576-f3492beea012 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:13.786589 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a972c31c-c101-48e4-a576-f3492beea012 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:13.814179 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:13.819931 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9aa57c-faef-4877-915b-ea4254955590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:13.820795 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9aa57c-faef-4877-915b-ea4254955590 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:13.821080 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9aa57c-faef-4877-915b-ea4254955590 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:13.821386 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9aa57c-faef-4877-915b-ea4254955590 None admin] PATH_INFO: `/v3/users/7aebda29924f48c882ffd12fc4448d17` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:13.822570 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9aa57c-faef-4877-915b-ea4254955590 None admin] RBAC: Authorizing `identity:get_user(user_id=7aebda29924f48c882ffd12fc4448d17)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:13.824016 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9aa57c-faef-4877-915b-ea4254955590 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:13.825692 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 574/1127] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:30:13 2022] GET /identity/v3/users/7aebda29924f48c882ffd12fc4448d17 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:30:13.831502 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb98e81-d6f4-45aa-b627-4a1defd9daff None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:13.933187 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb98e81-d6f4-45aa-b627-4a1defd9daff None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:13.946889 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:13.967008 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100a6c5-60b0-45fa-ae70-7f39cb0eb7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:13.967652 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100a6c5-60b0-45fa-ae70-7f39cb0eb7cf None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:13.967939 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100a6c5-60b0-45fa-ae70-7f39cb0eb7cf None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:13.968217 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100a6c5-60b0-45fa-ae70-7f39cb0eb7cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:14.042678 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b100a6c5-60b0-45fa-ae70-7f39cb0eb7cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:14.044131 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b100a6c5-60b0-45fa-ae70-7f39cb0eb7cf None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:14.149659 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 554/1128] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:30:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 16:30:14.157171 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9aa57c-faef-4877-915b-ea4254955590 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:14.262269 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9aa57c-faef-4877-915b-ea4254955590 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:14.296810 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650decb2-ae1b-46eb-9769-f5f050acc524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:14.297684 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650decb2-ae1b-46eb-9769-f5f050acc524 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:14.297968 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650decb2-ae1b-46eb-9769-f5f050acc524 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:14.298244 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650decb2-ae1b-46eb-9769-f5f050acc524 None admin] PATH_INFO: `/v3/projects/abba9b53b10946799633b36c9dee5f73` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:14.299411 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650decb2-ae1b-46eb-9769-f5f050acc524 None admin] RBAC: Authorizing `identity:get_project(project_id=abba9b53b10946799633b36c9dee5f73)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:14.300842 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650decb2-ae1b-46eb-9769-f5f050acc524 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:14.302643 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 575/1129] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:30:14 2022] GET /identity/v3/projects/abba9b53b10946799633b36c9dee5f73 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:30:14.308758 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100a6c5-60b0-45fa-ae70-7f39cb0eb7cf None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:14.323486 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:14.407989 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100a6c5-60b0-45fa-ae70-7f39cb0eb7cf None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:14.440776 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f10454-486d-47fe-9566-3a4629cf2597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:14.441536 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f10454-486d-47fe-9566-3a4629cf2597 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:14.441824 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f10454-486d-47fe-9566-3a4629cf2597 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:14.442108 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f10454-486d-47fe-9566-3a4629cf2597 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:14.442945 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f10454-486d-47fe-9566-3a4629cf2597 None admin] RBAC: Adding query filter params (role.id=50f62a33eb434682baa7529619fe14b0, scope.project.id=abba9b53b10946799633b36c9dee5f73, user.id=7aebda29924f48c882ffd12fc4448d17) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:30:14.443265 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f10454-486d-47fe-9566-3a4629cf2597 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:14.444697 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f10454-486d-47fe-9566-3a4629cf2597 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:14.450472 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 555/1130] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:30:14 2022] GET /identity/v3/role_assignments?user.id=7aebda29924f48c882ffd12fc4448d17&scope.project.id=abba9b53b10946799633b36c9dee5f73&role.id=50f62a33eb434682baa7529619fe14b0 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:30:16.815981 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ec524-e6c7-4988-bd9b-d66e18894420 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:16.816519 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ec524-e6c7-4988-bd9b-d66e18894420 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:16.816519 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ec524-e6c7-4988-bd9b-d66e18894420 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:16.817919 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 576/1131] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:30:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:30:16.823432 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fadc3a9-54dc-421f-9d3f-5861a2a5dd1d None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:16.823858 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fadc3a9-54dc-421f-9d3f-5861a2a5dd1d None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:16.824000 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fadc3a9-54dc-421f-9d3f-5861a2a5dd1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:16.841472 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-2fadc3a9-54dc-421f-9d3f-5861a2a5dd1d None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:30:16.842678 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-2fadc3a9-54dc-421f-9d3f-5861a2a5dd1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:30:16.916827 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 556/1132] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:30:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:30:16.940580 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817ec524-e6c7-4988-bd9b-d66e18894420 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:17.019744 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:17.054166 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817ec524-e6c7-4988-bd9b-d66e18894420 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:17.091089 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c226ddaf-5e7a-4ec3-994e-b18f774d4e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:17.091806 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226ddaf-5e7a-4ec3-994e-b18f774d4e9d None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:17.092095 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226ddaf-5e7a-4ec3-994e-b18f774d4e9d None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:17.092379 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226ddaf-5e7a-4ec3-994e-b18f774d4e9d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:17.093597 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c226ddaf-5e7a-4ec3-994e-b18f774d4e9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:17.095050 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c226ddaf-5e7a-4ec3-994e-b18f774d4e9d None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:17.096679 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 577/1133] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:30:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:30:17.102463 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fadc3a9-54dc-421f-9d3f-5861a2a5dd1d None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:17.112949 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:17.199751 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fadc3a9-54dc-421f-9d3f-5861a2a5dd1d None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:17.231463 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06ad151-be18-43a8-899d-8384e6b9c22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:17.232625 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ad151-be18-43a8-899d-8384e6b9c22c None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:17.232910 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ad151-be18-43a8-899d-8384e6b9c22c None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:17.233212 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ad151-be18-43a8-899d-8384e6b9c22c None admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:17.233919 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06ad151-be18-43a8-899d-8384e6b9c22c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:17.235351 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06ad151-be18-43a8-899d-8384e6b9c22c None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:17.259340 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 557/1134] 10.222.0.36 () {60 vars in 1141 bytes} [Thu May 19 16:30:17 2022] POST /identity/v3/projects => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:30:19.509847 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932760dd-25a0-4191-9dbf-77f3d12e4717 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:19.510360 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932760dd-25a0-4191-9dbf-77f3d12e4717 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:19.510455 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932760dd-25a0-4191-9dbf-77f3d12e4717 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:19.511770 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 578/1135] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:30:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:30:19.516869 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f9f6c-71b7-4a4c-9932-68b5d634e7d7 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:19.517322 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f9f6c-71b7-4a4c-9932-68b5d634e7d7 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:19.517477 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f9f6c-71b7-4a4c-9932-68b5d634e7d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:19.534106 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-424f9f6c-71b7-4a4c-9932-68b5d634e7d7 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:30:19.535281 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-424f9f6c-71b7-4a4c-9932-68b5d634e7d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:30:19.590168 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:19.602006 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 558/1136] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:30:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 16:30:19.627247 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932760dd-25a0-4191-9dbf-77f3d12e4717 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:19.724651 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932760dd-25a0-4191-9dbf-77f3d12e4717 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:19.768901 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa47297a-fd63-4e42-bbe5-9204916ebc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:19.769664 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47297a-fd63-4e42-bbe5-9204916ebc54 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:19.769949 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47297a-fd63-4e42-bbe5-9204916ebc54 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:19.770222 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47297a-fd63-4e42-bbe5-9204916ebc54 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:19.771753 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa47297a-fd63-4e42-bbe5-9204916ebc54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:19.773217 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa47297a-fd63-4e42-bbe5-9204916ebc54 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:19.774825 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 579/1137] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:30:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:30:19.781059 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f9f6c-71b7-4a4c-9932-68b5d634e7d7 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:19.876303 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f9f6c-71b7-4a4c-9932-68b5d634e7d7 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:19.909669 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63203ef-8002-4b59-9363-aae432853dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:19.910341 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63203ef-8002-4b59-9363-aae432853dd3 None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:19.910626 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63203ef-8002-4b59-9363-aae432853dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:19.910903 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63203ef-8002-4b59-9363-aae432853dd3 None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:19.911614 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63203ef-8002-4b59-9363-aae432853dd3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:19.913048 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63203ef-8002-4b59-9363-aae432853dd3 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:19.938047 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 559/1138] 10.222.0.36 () {60 vars in 1132 bytes} [Thu May 19 16:30:19 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:30:22.178804 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ab423-e4eb-4f5a-9ac3-fd03ed8a2561 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:22.179316 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ab423-e4eb-4f5a-9ac3-fd03ed8a2561 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:22.179316 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ab423-e4eb-4f5a-9ac3-fd03ed8a2561 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:22.180653 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 580/1139] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:30:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:30:22.186016 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a7e9f-7852-49e7-bc09-dc1811a316d8 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:22.186417 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a7e9f-7852-49e7-bc09-dc1811a316d8 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:22.186580 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a7e9f-7852-49e7-bc09-dc1811a316d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:22.202956 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-5b9a7e9f-7852-49e7-bc09-dc1811a316d8 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:30:22.204137 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-5b9a7e9f-7852-49e7-bc09-dc1811a316d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:30:22.245743 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:22.270373 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 560/1140] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:30:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:30:22.294638 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ab423-e4eb-4f5a-9ac3-fd03ed8a2561 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:22.297489 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:22.393768 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ab423-e4eb-4f5a-9ac3-fd03ed8a2561 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:22.427189 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ce4444-7baf-4c48-9712-1336739b5e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:22.427962 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ce4444-7baf-4c48-9712-1336739b5e8c None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:22.428248 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ce4444-7baf-4c48-9712-1336739b5e8c None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:22.428523 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ce4444-7baf-4c48-9712-1336739b5e8c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:22.438313 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ce4444-7baf-4c48-9712-1336739b5e8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:30:22.445700 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-c5ce4444-7baf-4c48-9712-1336739b5e8c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:30:22.445700 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:30:22.445700 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:30:22.445700 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:30:22.445700 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:30:22.445700 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:30:22.445700 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:30:22.445700 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:30:22.445700 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:30:22.445700 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:30:22.445700 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:30:22.445700 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:30:22.445700 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:30:22.445700 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:30:22.445700 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:30:22.445700 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:30:22.446775 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:30:22.446775 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:30:22.446775 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:30:22.446775 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:30:22.446775 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:30:22.446775 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:30:22.446775 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:30:22.446775 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:30:22.446775 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:30:22.446775 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:30:22.446775 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:30:22.446775 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:30:22.446775 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:30:22.446775 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:30:22.446775 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:30:22.446775 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:30:22.447939 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:30:22.447939 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:30:22.447939 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:30:22.447939 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:30:22.447939 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:30:22.447939 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:30:22.447939 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:30:22.447939 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:30:22.447939 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:30:22.447939 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:30:22.447939 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:30:22.447939 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:30:22.447939 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 581/1141] 10.222.0.36 () {56 vars in 1096 bytes} [Thu May 19 16:30:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:30:22.453425 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9a7e9f-7852-49e7-bc09-dc1811a316d8 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:22.551711 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9a7e9f-7852-49e7-bc09-dc1811a316d8 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:22.586169 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de6cbd-058a-4c5e-aa59-b98b57c46cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:22.586846 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de6cbd-058a-4c5e-aa59-b98b57c46cbb None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:22.587133 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de6cbd-058a-4c5e-aa59-b98b57c46cbb None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:22.587421 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de6cbd-058a-4c5e-aa59-b98b57c46cbb None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:22.588179 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19de6cbd-058a-4c5e-aa59-b98b57c46cbb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:30:22.588492 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19de6cbd-058a-4c5e-aa59-b98b57c46cbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:22.589917 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19de6cbd-058a-4c5e-aa59-b98b57c46cbb None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:22.598700 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 561/1142] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:30:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:30:22.604585 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ce4444-7baf-4c48-9712-1336739b5e8c None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:22.666818 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:22.705084 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ce4444-7baf-4c48-9712-1336739b5e8c None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:22.738284 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7703c10-4182-4d99-bb1d-d82a82b3244c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:22.739157 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7703c10-4182-4d99-bb1d-d82a82b3244c None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:22.739438 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7703c10-4182-4d99-bb1d-d82a82b3244c None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:22.739712 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7703c10-4182-4d99-bb1d-d82a82b3244c None admin] PATH_INFO: `/v3/users/8bac5cb167f5450c9e9483c775a6a7a1` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:22.753780 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7703c10-4182-4d99-bb1d-d82a82b3244c None admin] RBAC: Authorizing `identity:get_user(user_id=8bac5cb167f5450c9e9483c775a6a7a1)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:22.755261 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7703c10-4182-4d99-bb1d-d82a82b3244c None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:22.756970 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 582/1143] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:30:22 2022] GET /identity/v3/users/8bac5cb167f5450c9e9483c775a6a7a1 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:30:22.765054 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de6cbd-058a-4c5e-aa59-b98b57c46cbb None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:22.806973 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:22.864431 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de6cbd-058a-4c5e-aa59-b98b57c46cbb None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:22.899738 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db9dba9-efe9-4c1c-824e-39a2174118e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:22.900391 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db9dba9-efe9-4c1c-824e-39a2174118e6 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:22.900676 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db9dba9-efe9-4c1c-824e-39a2174118e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:22.900959 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db9dba9-efe9-4c1c-824e-39a2174118e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:22.967722 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db9dba9-efe9-4c1c-824e-39a2174118e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:22.969100 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db9dba9-efe9-4c1c-824e-39a2174118e6 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:23.005916 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:23.065489 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 562/1144] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:30:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 16:30:23.071324 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7703c10-4182-4d99-bb1d-d82a82b3244c None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:23.171161 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7703c10-4182-4d99-bb1d-d82a82b3244c None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:23.192189 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:23.206576 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e70ea09-9f65-4e7e-918d-55b76e825f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:23.207439 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70ea09-9f65-4e7e-918d-55b76e825f6d None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:23.207725 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70ea09-9f65-4e7e-918d-55b76e825f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:23.208005 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70ea09-9f65-4e7e-918d-55b76e825f6d None admin] PATH_INFO: `/v3/projects/88dcf2e52961411e8abbdd9dd43ed6f2` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:23.209229 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e70ea09-9f65-4e7e-918d-55b76e825f6d None admin] RBAC: Authorizing `identity:get_project(project_id=88dcf2e52961411e8abbdd9dd43ed6f2)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:23.210684 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e70ea09-9f65-4e7e-918d-55b76e825f6d None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:23.212398 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 583/1145] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:30:23 2022] GET /identity/v3/projects/88dcf2e52961411e8abbdd9dd43ed6f2 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:30:23.218791 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db9dba9-efe9-4c1c-824e-39a2174118e6 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:23.319800 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db9dba9-efe9-4c1c-824e-39a2174118e6 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:23.352927 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40292c7c-c4de-4b69-8902-d9eb34591f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:23.353622 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40292c7c-c4de-4b69-8902-d9eb34591f5a None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:23.353908 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40292c7c-c4de-4b69-8902-d9eb34591f5a None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:23.354188 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40292c7c-c4de-4b69-8902-d9eb34591f5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:23.355019 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40292c7c-c4de-4b69-8902-d9eb34591f5a None admin] RBAC: Adding query filter params (role.id=e2915fe109244a07a9be96bd2b219fdd, scope.project.id=88dcf2e52961411e8abbdd9dd43ed6f2, user.id=8bac5cb167f5450c9e9483c775a6a7a1) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:30:23.355331 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40292c7c-c4de-4b69-8902-d9eb34591f5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:23.357299 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40292c7c-c4de-4b69-8902-d9eb34591f5a None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:23.364890 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 563/1146] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:30:23 2022] GET /identity/v3/role_assignments?user.id=8bac5cb167f5450c9e9483c775a6a7a1&scope.project.id=88dcf2e52961411e8abbdd9dd43ed6f2&role.id=e2915fe109244a07a9be96bd2b219fdd => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:30:25.582062 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4df5c4-37fc-408e-aa17-5bda0891d13a None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:25.582591 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4df5c4-37fc-408e-aa17-5bda0891d13a None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:25.582704 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4df5c4-37fc-408e-aa17-5bda0891d13a None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:25.584000 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 584/1147] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:30:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:30:25.589325 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65edfcd8-a478-41f7-95bd-08297ca1e9d4 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:25.589725 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65edfcd8-a478-41f7-95bd-08297ca1e9d4 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:25.589898 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65edfcd8-a478-41f7-95bd-08297ca1e9d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:25.594926 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:25.607968 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-65edfcd8-a478-41f7-95bd-08297ca1e9d4 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:30:25.609136 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-65edfcd8-a478-41f7-95bd-08297ca1e9d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:30:25.676629 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 564/1148] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:30:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:30:25.700503 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4df5c4-37fc-408e-aa17-5bda0891d13a None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:25.809502 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4df5c4-37fc-408e-aa17-5bda0891d13a None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:25.842970 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a104e-4166-45b9-8d37-19ba2c4805b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:25.843685 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a104e-4166-45b9-8d37-19ba2c4805b1 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:25.843971 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a104e-4166-45b9-8d37-19ba2c4805b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:25.844249 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a104e-4166-45b9-8d37-19ba2c4805b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:25.854171 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56a104e-4166-45b9-8d37-19ba2c4805b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:30:25.862411 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-a56a104e-4166-45b9-8d37-19ba2c4805b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:30:25.862411 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:30:25.862411 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:30:25.862411 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:30:25.862411 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:30:25.862411 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:30:25.862411 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:30:25.862411 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:30:25.862411 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:30:25.862411 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:30:25.862411 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:30:25.862411 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:30:25.862411 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:30:25.862411 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:30:25.862411 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:30:25.862411 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:30:25.863629 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:30:25.863629 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:30:25.863629 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:30:25.863629 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:30:25.863629 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:30:25.863629 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:30:25.863629 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:30:25.863629 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:30:25.863629 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:30:25.863629 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:30:25.863629 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:30:25.863629 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:30:25.863629 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:30:25.863629 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:30:25.863629 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:30:25.863629 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:30:25.864584 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:30:25.864584 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:30:25.864584 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:30:25.864584 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:30:25.864584 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:30:25.864584 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:30:25.864584 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:30:25.864584 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:30:25.864584 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:30:25.864584 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:30:25.864584 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:30:25.864584 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:30:25.864584 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 585/1149] 10.222.0.36 () {56 vars in 1096 bytes} [Thu May 19 16:30:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:30:25.869647 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65edfcd8-a478-41f7-95bd-08297ca1e9d4 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:25.970241 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65edfcd8-a478-41f7-95bd-08297ca1e9d4 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:25.998939 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:26.004238 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa9eb4-f8b9-4b52-8283-a4e126c682f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:26.004966 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa9eb4-f8b9-4b52-8283-a4e126c682f9 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:26.005217 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa9eb4-f8b9-4b52-8283-a4e126c682f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:26.005499 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa9eb4-f8b9-4b52-8283-a4e126c682f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:26.006243 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aa9eb4-f8b9-4b52-8283-a4e126c682f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:30:26.006552 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aa9eb4-f8b9-4b52-8283-a4e126c682f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:26.007929 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aa9eb4-f8b9-4b52-8283-a4e126c682f9 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:26.015881 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 565/1150] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:30:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:30:26.022051 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a104e-4166-45b9-8d37-19ba2c4805b1 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:26.070552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:26.130554 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a104e-4166-45b9-8d37-19ba2c4805b1 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:26.164114 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee019b-0d22-461f-ad48-41628b9a6f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:26.165037 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee019b-0d22-461f-ad48-41628b9a6f66 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:26.165370 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee019b-0d22-461f-ad48-41628b9a6f66 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:26.165656 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee019b-0d22-461f-ad48-41628b9a6f66 None admin] PATH_INFO: `/v3/users/8bac5cb167f5450c9e9483c775a6a7a1` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:26.166882 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ee019b-0d22-461f-ad48-41628b9a6f66 None admin] RBAC: Authorizing `identity:get_user(user_id=8bac5cb167f5450c9e9483c775a6a7a1)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:26.168340 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ee019b-0d22-461f-ad48-41628b9a6f66 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:26.170059 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 586/1151] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:30:26 2022] GET /identity/v3/users/8bac5cb167f5450c9e9483c775a6a7a1 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:30:26.175957 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa9eb4-f8b9-4b52-8283-a4e126c682f9 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:26.273879 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa9eb4-f8b9-4b52-8283-a4e126c682f9 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:26.307122 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252407f3-9b5c-4e45-91af-236a9dfeeec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:26.307942 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252407f3-9b5c-4e45-91af-236a9dfeeec4 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:26.308226 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252407f3-9b5c-4e45-91af-236a9dfeeec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:26.308501 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252407f3-9b5c-4e45-91af-236a9dfeeec4 None admin] PATH_INFO: `/v3/projects/88dcf2e52961411e8abbdd9dd43ed6f2` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:26.310337 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252407f3-9b5c-4e45-91af-236a9dfeeec4 None admin] RBAC: Authorizing `identity:get_project(project_id=88dcf2e52961411e8abbdd9dd43ed6f2)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:26.311760 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252407f3-9b5c-4e45-91af-236a9dfeeec4 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:26.313471 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 566/1152] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:30:26 2022] GET /identity/v3/projects/88dcf2e52961411e8abbdd9dd43ed6f2 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:30:26.319545 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee019b-0d22-461f-ad48-41628b9a6f66 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:26.424000 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee019b-0d22-461f-ad48-41628b9a6f66 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:26.440636 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:26.458273 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc004c-9221-4d96-a355-d12a0b14f9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:26.459162 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc004c-9221-4d96-a355-d12a0b14f9a5 None admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:26.459442 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc004c-9221-4d96-a355-d12a0b14f9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:26.459719 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc004c-9221-4d96-a355-d12a0b14f9a5 None admin] PATH_INFO: `/v3/projects/88dcf2e52961411e8abbdd9dd43ed6f2/users/8bac5cb167f5450c9e9483c775a6a7a1/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:26.461715 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc004c-9221-4d96-a355-d12a0b14f9a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=88dcf2e52961411e8abbdd9dd43ed6f2, user_id=8bac5cb167f5450c9e9483c775a6a7a1, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:26.463162 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc004c-9221-4d96-a355-d12a0b14f9a5 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:26.475617 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 587/1153] 10.222.0.36 () {58 vars in 1439 bytes} [Thu May 19 16:30:26 2022] PUT /identity/v3/projects/88dcf2e52961411e8abbdd9dd43ed6f2/users/8bac5cb167f5450c9e9483c775a6a7a1/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:30:28.693723 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d054a-3555-4a5e-a3a6-4b8e4b92df58 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:28.694426 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d054a-3555-4a5e-a3a6-4b8e4b92df58 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:28.694426 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d054a-3555-4a5e-a3a6-4b8e4b92df58 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:28.695683 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 567/1154] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:30:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:30:28.701736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe76829-f34b-402b-8fe3-1956a8a6ea5f None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:28.702233 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe76829-f34b-402b-8fe3-1956a8a6ea5f None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:28.702323 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe76829-f34b-402b-8fe3-1956a8a6ea5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:28.720725 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-1fe76829-f34b-402b-8fe3-1956a8a6ea5f None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:30:28.721966 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-1fe76829-f34b-402b-8fe3-1956a8a6ea5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:30:28.792914 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 588/1155] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:30:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:30:28.816778 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d054a-3555-4a5e-a3a6-4b8e4b92df58 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:28.888417 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:28.935504 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d054a-3555-4a5e-a3a6-4b8e4b92df58 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:28.974682 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c77ad4-c3da-41b8-84d6-6924b039dcce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:28.975412 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c77ad4-c3da-41b8-84d6-6924b039dcce None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:28.975695 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c77ad4-c3da-41b8-84d6-6924b039dcce None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:28.975972 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c77ad4-c3da-41b8-84d6-6924b039dcce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:28.986699 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c77ad4-c3da-41b8-84d6-6924b039dcce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:30:28.994446 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-06c77ad4-c3da-41b8-84d6-6924b039dcce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:30:28.994446 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:30:28.994446 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:30:28.994446 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:30:28.994446 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:30:28.994446 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:30:28.994446 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:30:28.994446 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:30:28.994446 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:30:28.994446 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:30:28.994446 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:30:28.994446 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:30:28.994446 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:30:28.994446 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:30:28.994446 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:30:28.994446 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:30:28.996428 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:30:28.996428 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:30:28.996428 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:30:28.996428 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:30:28.996428 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:30:28.996428 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:30:28.996428 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:30:28.996428 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:30:28.996428 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:30:28.996428 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:30:28.996428 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:30:28.996428 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:30:28.996428 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:30:28.996428 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:30:28.996428 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:30:28.996428 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:30:28.998247 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:30:28.998247 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:30:28.998247 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:30:28.998247 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:30:28.998247 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:30:28.998247 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:30:28.998247 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:30:28.998247 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:30:28.998247 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:30:28.998247 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:30:28.998247 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:30:28.998247 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:30:28.998247 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 568/1156] 10.222.0.36 () {56 vars in 1096 bytes} [Thu May 19 16:30:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:30:29.001985 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe76829-f34b-402b-8fe3-1956a8a6ea5f None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:29.130789 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe76829-f34b-402b-8fe3-1956a8a6ea5f None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:29.139270 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:29.172310 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aecb9b9-a28c-4e77-9d17-6cb934b2465b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:29.173195 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aecb9b9-a28c-4e77-9d17-6cb934b2465b None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:29.173571 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aecb9b9-a28c-4e77-9d17-6cb934b2465b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:29.173912 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aecb9b9-a28c-4e77-9d17-6cb934b2465b None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:29.174745 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aecb9b9-a28c-4e77-9d17-6cb934b2465b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:30:29.175119 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aecb9b9-a28c-4e77-9d17-6cb934b2465b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:29.177128 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aecb9b9-a28c-4e77-9d17-6cb934b2465b None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:29.186600 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 589/1157] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:30:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:30:29.193241 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c77ad4-c3da-41b8-84d6-6924b039dcce None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:29.306974 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c77ad4-c3da-41b8-84d6-6924b039dcce None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:29.333922 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:29.347180 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df973a-ce13-444b-88d9-9ca376e382a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:29.348069 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df973a-ce13-444b-88d9-9ca376e382a1 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:29.348362 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df973a-ce13-444b-88d9-9ca376e382a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:29.349014 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df973a-ce13-444b-88d9-9ca376e382a1 None admin] PATH_INFO: `/v3/users/8bac5cb167f5450c9e9483c775a6a7a1` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:29.349909 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98df973a-ce13-444b-88d9-9ca376e382a1 None admin] RBAC: Authorizing `identity:get_user(user_id=8bac5cb167f5450c9e9483c775a6a7a1)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:29.351312 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98df973a-ce13-444b-88d9-9ca376e382a1 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:29.352918 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 569/1158] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:30:29 2022] GET /identity/v3/users/8bac5cb167f5450c9e9483c775a6a7a1 => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:30:29.358708 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aecb9b9-a28c-4e77-9d17-6cb934b2465b None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:29.471121 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aecb9b9-a28c-4e77-9d17-6cb934b2465b None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:29.507885 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28667769-2a1e-472d-964e-13beac43e797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:29.508545 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28667769-2a1e-472d-964e-13beac43e797 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:29.508834 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28667769-2a1e-472d-964e-13beac43e797 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:29.509114 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28667769-2a1e-472d-964e-13beac43e797 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:29.545405 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:29.581823 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28667769-2a1e-472d-964e-13beac43e797 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:29.583233 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28667769-2a1e-472d-964e-13beac43e797 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:29.686458 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 590/1159] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:30:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 16:30:29.692341 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df973a-ce13-444b-88d9-9ca376e382a1 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:29.798472 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df973a-ce13-444b-88d9-9ca376e382a1 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:29.833536 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8c4373-f9d0-45f0-8aa6-e1f2184a5ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:29.834336 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c4373-f9d0-45f0-8aa6-e1f2184a5ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:29.834623 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c4373-f9d0-45f0-8aa6-e1f2184a5ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:29.834902 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c4373-f9d0-45f0-8aa6-e1f2184a5ef1 None admin] PATH_INFO: `/v3/projects/88dcf2e52961411e8abbdd9dd43ed6f2` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:29.836095 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8c4373-f9d0-45f0-8aa6-e1f2184a5ef1 None admin] RBAC: Authorizing `identity:get_project(project_id=88dcf2e52961411e8abbdd9dd43ed6f2)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:29.837566 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8c4373-f9d0-45f0-8aa6-e1f2184a5ef1 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:29.839267 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 570/1160] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:30:29 2022] GET /identity/v3/projects/88dcf2e52961411e8abbdd9dd43ed6f2 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:30:29.845918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28667769-2a1e-472d-964e-13beac43e797 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:29.916771 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:29.957567 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28667769-2a1e-472d-964e-13beac43e797 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:29.992390 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5d770-2cf3-4008-8a48-9f011aab3ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:29.993055 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5d770-2cf3-4008-8a48-9f011aab3ff7 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:29.993439 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5d770-2cf3-4008-8a48-9f011aab3ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:29.993667 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5d770-2cf3-4008-8a48-9f011aab3ff7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:29.994508 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b5d770-2cf3-4008-8a48-9f011aab3ff7 None admin] RBAC: Adding query filter params (role.id=e2915fe109244a07a9be96bd2b219fdd, scope.project.id=88dcf2e52961411e8abbdd9dd43ed6f2, user.id=8bac5cb167f5450c9e9483c775a6a7a1) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:30:29.994829 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b5d770-2cf3-4008-8a48-9f011aab3ff7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:29.996281 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b5d770-2cf3-4008-8a48-9f011aab3ff7 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:30.002648 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 591/1161] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:30:29 2022] GET /identity/v3/role_assignments?user.id=8bac5cb167f5450c9e9483c775a6a7a1&scope.project.id=88dcf2e52961411e8abbdd9dd43ed6f2&role.id=e2915fe109244a07a9be96bd2b219fdd => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:30:32.261947 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71006cab-d7ce-406b-8a4e-f6df5acbf430 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:32.262445 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71006cab-d7ce-406b-8a4e-f6df5acbf430 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:32.262445 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71006cab-d7ce-406b-8a4e-f6df5acbf430 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:32.263857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 571/1162] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:30:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:30:32.269873 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c539b-62e5-466c-a13c-353ad61d9c2b None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:32.270321 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c539b-62e5-466c-a13c-353ad61d9c2b None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:32.270427 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c539b-62e5-466c-a13c-353ad61d9c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:32.287106 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-647c539b-62e5-466c-a13c-353ad61d9c2b None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:30:32.288303 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-647c539b-62e5-466c-a13c-353ad61d9c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:30:32.357003 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 592/1163] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:30:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:30:32.380093 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71006cab-d7ce-406b-8a4e-f6df5acbf430 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:32.443811 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:32.488006 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71006cab-d7ce-406b-8a4e-f6df5acbf430 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:32.523553 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9b07d4-daca-4437-8c8f-add3eab03b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:32.524139 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b07d4-daca-4437-8c8f-add3eab03b19 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:32.524425 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b07d4-daca-4437-8c8f-add3eab03b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:32.524704 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b07d4-daca-4437-8c8f-add3eab03b19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:32.623034 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9b07d4-daca-4437-8c8f-add3eab03b19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:32.624665 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9b07d4-daca-4437-8c8f-add3eab03b19 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:32.714974 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:32.760180 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 572/1164] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:30:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 16:30:32.766160 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647c539b-62e5-466c-a13c-353ad61d9c2b None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:32.831225 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:32.870878 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647c539b-62e5-466c-a13c-353ad61d9c2b None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:32.911090 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a2bfff-8393-4049-991a-d6cc8d6cbbd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:32.911908 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2bfff-8393-4049-991a-d6cc8d6cbbd0 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:32.912192 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2bfff-8393-4049-991a-d6cc8d6cbbd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:32.912467 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2bfff-8393-4049-991a-d6cc8d6cbbd0 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:32.919618 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a2bfff-8393-4049-991a-d6cc8d6cbbd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:32.921095 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a2bfff-8393-4049-991a-d6cc8d6cbbd0 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:32.935552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-76a2bfff-8393-4049-991a-d6cc8d6cbbd0 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 16:30:32.935552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:30:32.935552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 19 16:30:32.935552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 19 16:30:32.935552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 16:30:32.935552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 16:30:32.935552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 16:30:32.935552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 16:30:32.935552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 19 16:30:32.935552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:30:32.935552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 16:30:32.935552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:30:32.935552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:30:32.935552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:30:32.935552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:30:32.935552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:30:32.935552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:30:32.936804 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:30:32.936804 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:30:32.936804 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:30:32.936804 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:30:32.936804 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:30:32.936804 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:30:32.936804 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:30:32.936804 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:30:32.936804 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:30:32.936804 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:30:32.936804 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:30:32.936804 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:30:32.936804 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:30:32.936804 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:30:32.936804 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:30:32.936804 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:30:32.938093 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:30:32.938093 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:30:32.938093 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:30:32.938093 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:30:32.938093 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:30:32.938093 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:30:32.938093 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:30:32.938093 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:30:32.938093 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:30:32.938093 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:30:32.938093 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 19 16:30:32.938093 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:30:32.938093 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 16:30:32.938093 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:30:32.938093 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 593/1165] 10.222.0.36 () {56 vars in 1117 bytes} [Thu May 19 16:30:32 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:30:32.942999 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9b07d4-daca-4437-8c8f-add3eab03b19 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:33.065329 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9b07d4-daca-4437-8c8f-add3eab03b19 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:33.103277 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1918ac77-e8e5-4dfc-84ef-888b5bd9eafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:33.104046 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918ac77-e8e5-4dfc-84ef-888b5bd9eafc None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:33.104389 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918ac77-e8e5-4dfc-84ef-888b5bd9eafc None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:33.104733 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918ac77-e8e5-4dfc-84ef-888b5bd9eafc None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:33.106080 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1918ac77-e8e5-4dfc-84ef-888b5bd9eafc None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:30:33.106549 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1918ac77-e8e5-4dfc-84ef-888b5bd9eafc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:33.108623 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1918ac77-e8e5-4dfc-84ef-888b5bd9eafc None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:33.119165 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 573/1166] 10.222.0.36 () {56 vars in 1115 bytes} [Thu May 19 16:30:32 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:30:35.336636 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217fabba-ed36-4730-bc36-7997a11d8213 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:35.337191 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217fabba-ed36-4730-bc36-7997a11d8213 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:35.337191 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217fabba-ed36-4730-bc36-7997a11d8213 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:35.338515 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 594/1167] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:30:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:30:35.343827 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1576a8e-ad4f-4a02-8949-e0e7854d6e1e None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:35.344234 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1576a8e-ad4f-4a02-8949-e0e7854d6e1e None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:35.344399 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1576a8e-ad4f-4a02-8949-e0e7854d6e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:35.361600 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-f1576a8e-ad4f-4a02-8949-e0e7854d6e1e None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:30:35.362796 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-f1576a8e-ad4f-4a02-8949-e0e7854d6e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:30:35.379407 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:35.428644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 574/1168] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:30:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:30:35.451443 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217fabba-ed36-4730-bc36-7997a11d8213 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:35.556180 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217fabba-ed36-4730-bc36-7997a11d8213 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:35.579097 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:35.591450 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1b78db-17ed-4c03-abb7-a6ab00a511a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:35.592054 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b78db-17ed-4c03-abb7-a6ab00a511a1 None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:35.592343 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b78db-17ed-4c03-abb7-a6ab00a511a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:35.592625 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b78db-17ed-4c03-abb7-a6ab00a511a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:35.593405 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1b78db-17ed-4c03-abb7-a6ab00a511a1 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:35.594799 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1b78db-17ed-4c03-abb7-a6ab00a511a1 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:35.609369 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 595/1169] 10.222.0.36 () {60 vars in 1138 bytes} [Thu May 19 16:30:35 2022] POST /identity/v3/domains => generated 267 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:30:37.848750 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00132d5-8d41-429d-b1ea-62d3fb7a2d66 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:37.849328 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00132d5-8d41-429d-b1ea-62d3fb7a2d66 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:37.849328 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00132d5-8d41-429d-b1ea-62d3fb7a2d66 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:37.850723 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 575/1170] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:30:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:30:37.855992 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732dfd8f-3aa9-4687-9bce-03e048399402 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:37.856394 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732dfd8f-3aa9-4687-9bce-03e048399402 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:37.856549 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732dfd8f-3aa9-4687-9bce-03e048399402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:37.873805 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-732dfd8f-3aa9-4687-9bce-03e048399402 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:30:37.875025 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-732dfd8f-3aa9-4687-9bce-03e048399402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:30:37.941752 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 596/1171] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:30:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:30:37.964863 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00132d5-8d41-429d-b1ea-62d3fb7a2d66 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:38.068155 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00132d5-8d41-429d-b1ea-62d3fb7a2d66 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:38.100713 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4713a6cc-6a03-4131-8aba-1d34cbf69ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:38.101531 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4713a6cc-6a03-4131-8aba-1d34cbf69ae1 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:38.101812 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4713a6cc-6a03-4131-8aba-1d34cbf69ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:38.102086 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4713a6cc-6a03-4131-8aba-1d34cbf69ae1 None admin] PATH_INFO: `/v3/domains/98f634c7fea848ad8a0b7d5f849d5f9b` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:38.112634 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4713a6cc-6a03-4131-8aba-1d34cbf69ae1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=98f634c7fea848ad8a0b7d5f849d5f9b)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:38.114091 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4713a6cc-6a03-4131-8aba-1d34cbf69ae1 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:38.115712 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 576/1172] 10.222.0.36 () {56 vars in 1183 bytes} [Thu May 19 16:30:37 2022] GET /identity/v3/domains/98f634c7fea848ad8a0b7d5f849d5f9b => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:30:38.121608 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732dfd8f-3aa9-4687-9bce-03e048399402 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:38.223281 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732dfd8f-3aa9-4687-9bce-03e048399402 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:38.236254 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:38.257624 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1167a8c4-67da-4f9f-973a-35e5355315c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:38.258285 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167a8c4-67da-4f9f-973a-35e5355315c5 None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:38.258572 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167a8c4-67da-4f9f-973a-35e5355315c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:38.258854 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167a8c4-67da-4f9f-973a-35e5355315c5 None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:38.259561 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1167a8c4-67da-4f9f-973a-35e5355315c5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:38.261017 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1167a8c4-67da-4f9f-973a-35e5355315c5 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:38.286314 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 597/1173] 10.222.0.36 () {60 vars in 1141 bytes} [Thu May 19 16:30:38 2022] POST /identity/v3/projects => generated 361 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:30:40.566341 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7991cba-b205-4aa1-91ca-86c5b126d92e None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:40.566900 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7991cba-b205-4aa1-91ca-86c5b126d92e None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:40.566900 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7991cba-b205-4aa1-91ca-86c5b126d92e None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:40.568331 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 577/1174] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:30:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:30:40.573649 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d362f6-59bd-4acd-bb1a-5f8e4bef484e None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:40.574067 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d362f6-59bd-4acd-bb1a-5f8e4bef484e None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:40.574212 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d362f6-59bd-4acd-bb1a-5f8e4bef484e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:40.591661 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-e4d362f6-59bd-4acd-bb1a-5f8e4bef484e None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:30:40.592830 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e4d362f6-59bd-4acd-bb1a-5f8e4bef484e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:30:40.656930 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 598/1175] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:30:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:30:40.681254 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7991cba-b205-4aa1-91ca-86c5b126d92e None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:40.697005 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:40.781683 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7991cba-b205-4aa1-91ca-86c5b126d92e None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:40.814854 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f1ed03-5ceb-42ff-91d9-1c7b27a99ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:40.815704 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1ed03-5ceb-42ff-91d9-1c7b27a99ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:40.815988 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1ed03-5ceb-42ff-91d9-1c7b27a99ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:40.816266 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1ed03-5ceb-42ff-91d9-1c7b27a99ce3 None admin] PATH_INFO: `/v3/domains/98f634c7fea848ad8a0b7d5f849d5f9b` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:40.817609 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f1ed03-5ceb-42ff-91d9-1c7b27a99ce3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=98f634c7fea848ad8a0b7d5f849d5f9b)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:40.818996 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f1ed03-5ceb-42ff-91d9-1c7b27a99ce3 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:40.820604 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 578/1176] 10.222.0.36 () {56 vars in 1183 bytes} [Thu May 19 16:30:40 2022] GET /identity/v3/domains/98f634c7fea848ad8a0b7d5f849d5f9b => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:30:40.826965 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d362f6-59bd-4acd-bb1a-5f8e4bef484e None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:40.919616 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:40.927902 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d362f6-59bd-4acd-bb1a-5f8e4bef484e None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:40.962307 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d84eca-682e-4b4a-97c1-fe370a3f8e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:40.962969 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d84eca-682e-4b4a-97c1-fe370a3f8e46 None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:40.963251 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d84eca-682e-4b4a-97c1-fe370a3f8e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:40.963528 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d84eca-682e-4b4a-97c1-fe370a3f8e46 None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:40.964226 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d84eca-682e-4b4a-97c1-fe370a3f8e46 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:40.965754 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d84eca-682e-4b4a-97c1-fe370a3f8e46 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:40.984369 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 599/1177] 10.222.0.36 () {60 vars in 1132 bytes} [Thu May 19 16:30:40 2022] POST /identity/v3/users => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:30:43.240287 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da311a-2833-4c9f-b7e7-94e0e522d741 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:43.240795 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da311a-2833-4c9f-b7e7-94e0e522d741 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:43.240795 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da311a-2833-4c9f-b7e7-94e0e522d741 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:43.242262 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 579/1178] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:30:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:30:43.248140 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5714ac3e-46ca-4059-ad30-9f7f5ef68f4d None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:43.248575 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5714ac3e-46ca-4059-ad30-9f7f5ef68f4d None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:43.248698 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5714ac3e-46ca-4059-ad30-9f7f5ef68f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:43.265680 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-5714ac3e-46ca-4059-ad30-9f7f5ef68f4d None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:30:43.266865 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-5714ac3e-46ca-4059-ad30-9f7f5ef68f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:30:43.330629 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 600/1179] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:30:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:30:43.353657 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da311a-2833-4c9f-b7e7-94e0e522d741 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:43.434833 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:43.455119 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da311a-2833-4c9f-b7e7-94e0e522d741 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:43.487905 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52e280c-d261-4307-ad02-fa53cd9756c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:43.488687 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e280c-d261-4307-ad02-fa53cd9756c8 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:43.488971 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e280c-d261-4307-ad02-fa53cd9756c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:43.489274 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e280c-d261-4307-ad02-fa53cd9756c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:43.499850 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52e280c-d261-4307-ad02-fa53cd9756c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:30:43.507275 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-d52e280c-d261-4307-ad02-fa53cd9756c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:30:43.507275 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:30:43.507275 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:30:43.507275 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:30:43.507275 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:30:43.507275 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:30:43.507275 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:30:43.507275 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:30:43.507275 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:30:43.507275 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:30:43.507275 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:30:43.507275 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:30:43.507275 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:30:43.507275 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:30:43.507275 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:30:43.507275 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:30:43.508364 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:30:43.508364 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:30:43.508364 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:30:43.508364 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:30:43.508364 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:30:43.508364 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:30:43.508364 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:30:43.508364 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:30:43.508364 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:30:43.508364 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:30:43.508364 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:30:43.508364 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:30:43.508364 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:30:43.508364 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:30:43.508364 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:30:43.508364 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:30:43.509438 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:30:43.509438 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:30:43.509438 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:30:43.509438 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:30:43.509438 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:30:43.509438 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:30:43.509438 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:30:43.509438 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:30:43.509438 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:30:43.509438 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:30:43.509438 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:30:43.509438 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:30:43.509438 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 580/1180] 10.222.0.36 () {56 vars in 1096 bytes} [Thu May 19 16:30:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:30:43.514580 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5714ac3e-46ca-4059-ad30-9f7f5ef68f4d None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:43.608827 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:43.632932 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5714ac3e-46ca-4059-ad30-9f7f5ef68f4d None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:43.667975 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf3b7c7-4d0f-4b54-b9ec-20384e378cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:43.669141 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3b7c7-4d0f-4b54-b9ec-20384e378cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:43.669452 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3b7c7-4d0f-4b54-b9ec-20384e378cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:43.669727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3b7c7-4d0f-4b54-b9ec-20384e378cd4 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:43.670475 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf3b7c7-4d0f-4b54-b9ec-20384e378cd4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:30:43.670783 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf3b7c7-4d0f-4b54-b9ec-20384e378cd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:43.672165 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf3b7c7-4d0f-4b54-b9ec-20384e378cd4 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:43.679965 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 601/1181] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:30:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:30:43.686513 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52e280c-d261-4307-ad02-fa53cd9756c8 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:43.782906 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52e280c-d261-4307-ad02-fa53cd9756c8 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:43.814911 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f83874c-dd1b-454b-a168-99f681c877b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:43.815765 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f83874c-dd1b-454b-a168-99f681c877b9 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:43.816047 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f83874c-dd1b-454b-a168-99f681c877b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:43.816323 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f83874c-dd1b-454b-a168-99f681c877b9 None admin] PATH_INFO: `/v3/users/d14a75a485fd423bab1201b7d7847aa3` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:43.826264 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:43.829638 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f83874c-dd1b-454b-a168-99f681c877b9 None admin] RBAC: Authorizing `identity:get_user(user_id=d14a75a485fd423bab1201b7d7847aa3)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:43.831303 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f83874c-dd1b-454b-a168-99f681c877b9 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:43.833005 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 581/1182] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:30:43 2022] GET /identity/v3/users/d14a75a485fd423bab1201b7d7847aa3 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:30:43.838836 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf3b7c7-4d0f-4b54-b9ec-20384e378cd4 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:43.966986 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf3b7c7-4d0f-4b54-b9ec-20384e378cd4 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:44.000587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58df8039-3dfe-4294-843e-6c2709cfc49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:44.001269 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df8039-3dfe-4294-843e-6c2709cfc49f None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:44.001557 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df8039-3dfe-4294-843e-6c2709cfc49f None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:44.001846 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df8039-3dfe-4294-843e-6c2709cfc49f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:44.014530 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:44.077386 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58df8039-3dfe-4294-843e-6c2709cfc49f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:44.078793 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58df8039-3dfe-4294-843e-6c2709cfc49f None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:44.178737 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 602/1183] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:30:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 16:30:44.184773 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f83874c-dd1b-454b-a168-99f681c877b9 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:44.295865 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f83874c-dd1b-454b-a168-99f681c877b9 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:44.328731 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929a2d10-d2c7-4777-93c0-f599813d44bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:44.329542 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a2d10-d2c7-4777-93c0-f599813d44bb None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:44.329827 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a2d10-d2c7-4777-93c0-f599813d44bb None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:44.330103 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a2d10-d2c7-4777-93c0-f599813d44bb None admin] PATH_INFO: `/v3/projects/35540a5a3ee84511b8d8cab14689a3b9` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:44.331281 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929a2d10-d2c7-4777-93c0-f599813d44bb None admin] RBAC: Authorizing `identity:get_project(project_id=35540a5a3ee84511b8d8cab14689a3b9)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:44.332723 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929a2d10-d2c7-4777-93c0-f599813d44bb None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:44.334463 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 582/1184] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:30:44 2022] GET /identity/v3/projects/35540a5a3ee84511b8d8cab14689a3b9 => generated 361 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:30:44.341102 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58df8039-3dfe-4294-843e-6c2709cfc49f None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:44.382697 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:44.442658 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58df8039-3dfe-4294-843e-6c2709cfc49f None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:44.480375 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8cb48b-6a48-436a-9bd9-7a512376f72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:44.481037 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8cb48b-6a48-436a-9bd9-7a512376f72b None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:44.481353 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8cb48b-6a48-436a-9bd9-7a512376f72b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:44.481632 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8cb48b-6a48-436a-9bd9-7a512376f72b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:44.482830 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8cb48b-6a48-436a-9bd9-7a512376f72b None admin] RBAC: Adding query filter params (role.id=e2915fe109244a07a9be96bd2b219fdd, scope.project.id=35540a5a3ee84511b8d8cab14689a3b9, user.id=d14a75a485fd423bab1201b7d7847aa3) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:30:44.482830 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8cb48b-6a48-436a-9bd9-7a512376f72b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:44.484250 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8cb48b-6a48-436a-9bd9-7a512376f72b None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:44.489990 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 603/1185] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:30:44 2022] GET /identity/v3/role_assignments?user.id=d14a75a485fd423bab1201b7d7847aa3&scope.project.id=35540a5a3ee84511b8d8cab14689a3b9&role.id=e2915fe109244a07a9be96bd2b219fdd => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:30:46.700710 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d8d4d-a6d4-49e2-be1b-c28dacb1a20a None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:46.701284 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d8d4d-a6d4-49e2-be1b-c28dacb1a20a None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:46.701284 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d8d4d-a6d4-49e2-be1b-c28dacb1a20a None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:46.702658 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 583/1186] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:30:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:30:46.708119 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a79dc-03bd-4dca-9e21-15d8180f3877 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:46.708556 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a79dc-03bd-4dca-9e21-15d8180f3877 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:46.708674 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a79dc-03bd-4dca-9e21-15d8180f3877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:46.726098 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-c70a79dc-03bd-4dca-9e21-15d8180f3877 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:30:46.727314 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-c70a79dc-03bd-4dca-9e21-15d8180f3877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:30:46.798242 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 604/1187] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:30:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:30:46.820661 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d8d4d-a6d4-49e2-be1b-c28dacb1a20a None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:46.888793 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:46.930878 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d8d4d-a6d4-49e2-be1b-c28dacb1a20a None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:46.964499 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca45679-abea-4f5d-a5b6-18d158740d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:46.965784 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca45679-abea-4f5d-a5b6-18d158740d60 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:46.966074 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca45679-abea-4f5d-a5b6-18d158740d60 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:46.966354 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca45679-abea-4f5d-a5b6-18d158740d60 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:46.976850 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca45679-abea-4f5d-a5b6-18d158740d60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:30:46.985066 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-0ca45679-abea-4f5d-a5b6-18d158740d60 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:30:46.985066 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:30:46.985066 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:30:46.985066 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:30:46.985066 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:30:46.985066 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:30:46.985066 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:30:46.985066 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:30:46.985066 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:30:46.985066 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:30:46.985066 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:30:46.985066 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:30:46.985066 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:30:46.985066 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:30:46.985066 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:30:46.985066 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:30:46.986248 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:30:46.986248 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:30:46.986248 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:30:46.986248 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:30:46.986248 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:30:46.986248 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:30:46.986248 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:30:46.986248 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:30:46.986248 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:30:46.986248 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:30:46.986248 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:30:46.986248 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:30:46.986248 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:30:46.986248 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:30:46.986248 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:30:46.986248 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:30:46.987128 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:30:46.987128 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:30:46.987128 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:30:46.987128 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:30:46.987128 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:30:46.987128 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:30:46.987128 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:30:46.987128 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:30:46.987128 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:30:46.987128 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:30:46.987128 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:30:46.987128 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:30:46.987128 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 584/1188] 10.222.0.36 () {56 vars in 1096 bytes} [Thu May 19 16:30:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:30:46.992306 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70a79dc-03bd-4dca-9e21-15d8180f3877 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:47.026222 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:47.091066 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70a79dc-03bd-4dca-9e21-15d8180f3877 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:47.124716 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93263b4c-7a19-4319-9fb7-09f221864633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:47.125405 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93263b4c-7a19-4319-9fb7-09f221864633 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:47.125692 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93263b4c-7a19-4319-9fb7-09f221864633 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:47.125968 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93263b4c-7a19-4319-9fb7-09f221864633 None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:47.126748 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93263b4c-7a19-4319-9fb7-09f221864633 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:30:47.127057 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93263b4c-7a19-4319-9fb7-09f221864633 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:47.128434 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93263b4c-7a19-4319-9fb7-09f221864633 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:47.136712 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 605/1189] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:30:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:30:47.143557 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca45679-abea-4f5d-a5b6-18d158740d60 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:47.244634 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca45679-abea-4f5d-a5b6-18d158740d60 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:47.269802 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:47.279269 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ff6f09-4315-4a04-ba36-49af8d072daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:47.280119 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff6f09-4315-4a04-ba36-49af8d072daf None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:47.280402 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff6f09-4315-4a04-ba36-49af8d072daf None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:47.280679 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff6f09-4315-4a04-ba36-49af8d072daf None admin] PATH_INFO: `/v3/users/d14a75a485fd423bab1201b7d7847aa3` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:47.281890 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ff6f09-4315-4a04-ba36-49af8d072daf None admin] RBAC: Authorizing `identity:get_user(user_id=d14a75a485fd423bab1201b7d7847aa3)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:47.283332 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ff6f09-4315-4a04-ba36-49af8d072daf None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:47.284956 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 585/1190] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:30:47 2022] GET /identity/v3/users/d14a75a485fd423bab1201b7d7847aa3 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:30:47.290824 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93263b4c-7a19-4319-9fb7-09f221864633 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:47.393208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:47.397560 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93263b4c-7a19-4319-9fb7-09f221864633 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:47.431929 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd94fcb-a0c2-4828-a952-a754923df582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:47.432808 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd94fcb-a0c2-4828-a952-a754923df582 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:47.433094 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd94fcb-a0c2-4828-a952-a754923df582 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:47.433405 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd94fcb-a0c2-4828-a952-a754923df582 None admin] PATH_INFO: `/v3/projects/35540a5a3ee84511b8d8cab14689a3b9` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:47.434659 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd94fcb-a0c2-4828-a952-a754923df582 None admin] RBAC: Authorizing `identity:get_project(project_id=35540a5a3ee84511b8d8cab14689a3b9)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:47.436124 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd94fcb-a0c2-4828-a952-a754923df582 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:47.437916 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 606/1191] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:30:47 2022] GET /identity/v3/projects/35540a5a3ee84511b8d8cab14689a3b9 => generated 361 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:30:47.446178 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ff6f09-4315-4a04-ba36-49af8d072daf None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:47.550444 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ff6f09-4315-4a04-ba36-49af8d072daf None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:47.584785 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8afe18-dfcf-4455-88fb-e20af853b92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:47.585707 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8afe18-dfcf-4455-88fb-e20af853b92f None admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:47.585990 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8afe18-dfcf-4455-88fb-e20af853b92f None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:47.586268 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8afe18-dfcf-4455-88fb-e20af853b92f None admin] PATH_INFO: `/v3/projects/35540a5a3ee84511b8d8cab14689a3b9/users/d14a75a485fd423bab1201b7d7847aa3/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:47.588313 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8afe18-dfcf-4455-88fb-e20af853b92f None admin] RBAC: Authorizing `identity:create_grant(project_id=35540a5a3ee84511b8d8cab14689a3b9, user_id=d14a75a485fd423bab1201b7d7847aa3, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:47.589797 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8afe18-dfcf-4455-88fb-e20af853b92f None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:47.603896 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 586/1192] 10.222.0.36 () {58 vars in 1439 bytes} [Thu May 19 16:30:47 2022] PUT /identity/v3/projects/35540a5a3ee84511b8d8cab14689a3b9/users/d14a75a485fd423bab1201b7d7847aa3/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:30:49.834937 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b96fca-a6f4-40b8-a30d-1d2801fedf23 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:49.835494 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b96fca-a6f4-40b8-a30d-1d2801fedf23 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:49.835494 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b96fca-a6f4-40b8-a30d-1d2801fedf23 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:49.836864 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 607/1193] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:30:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:30:49.842311 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20262e5b-e612-49c0-b7e1-bcbf740a65bb None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:49.842762 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20262e5b-e612-49c0-b7e1-bcbf740a65bb None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:49.842888 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20262e5b-e612-49c0-b7e1-bcbf740a65bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:49.863189 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-20262e5b-e612-49c0-b7e1-bcbf740a65bb None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:30:49.864369 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-20262e5b-e612-49c0-b7e1-bcbf740a65bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:30:49.914115 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:49.934985 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 587/1194] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:30:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 16:30:49.958788 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b96fca-a6f4-40b8-a30d-1d2801fedf23 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:50.066151 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b96fca-a6f4-40b8-a30d-1d2801fedf23 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:50.101442 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5bac14-09c1-4fce-befe-b36337e51a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:50.102155 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5bac14-09c1-4fce-befe-b36337e51a1d None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:50.102437 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5bac14-09c1-4fce-befe-b36337e51a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:50.102714 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5bac14-09c1-4fce-befe-b36337e51a1d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:50.114100 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5bac14-09c1-4fce-befe-b36337e51a1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:30:50.122271 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-df5bac14-09c1-4fce-befe-b36337e51a1d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:30:50.122271 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:30:50.122271 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:30:50.122271 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:30:50.122271 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:30:50.122271 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:30:50.122271 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:30:50.122271 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:30:50.122271 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:30:50.122271 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:30:50.122271 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:30:50.122271 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:30:50.122271 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:30:50.122271 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:30:50.122271 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:30:50.122271 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:30:50.123453 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:30:50.123453 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:30:50.123453 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:30:50.123453 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:30:50.123453 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:30:50.123453 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:30:50.123453 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:30:50.123453 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:30:50.123453 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:30:50.123453 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:30:50.123453 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:30:50.123453 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:30:50.123453 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:30:50.123453 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:30:50.123453 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:30:50.123453 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:30:50.124351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:30:50.124351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:30:50.124351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:30:50.124351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:30:50.124351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:30:50.124351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:30:50.124351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:30:50.124351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:30:50.124351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:30:50.124351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:30:50.124351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:30:50.124351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:30:50.124351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 608/1195] 10.222.0.36 () {56 vars in 1096 bytes} [Thu May 19 16:30:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:30:50.132444 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20262e5b-e612-49c0-b7e1-bcbf740a65bb None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:50.277310 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20262e5b-e612-49c0-b7e1-bcbf740a65bb None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:50.310912 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abef6d8-b4a0-4083-ab0d-4d4c32c4c298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:50.311583 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abef6d8-b4a0-4083-ab0d-4d4c32c4c298 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:50.311866 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abef6d8-b4a0-4083-ab0d-4d4c32c4c298 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:50.312139 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abef6d8-b4a0-4083-ab0d-4d4c32c4c298 None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:50.312887 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abef6d8-b4a0-4083-ab0d-4d4c32c4c298 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:30:50.313217 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abef6d8-b4a0-4083-ab0d-4d4c32c4c298 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:50.314575 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abef6d8-b4a0-4083-ab0d-4d4c32c4c298 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:50.322404 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 588/1196] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:30:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:30:50.328191 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5bac14-09c1-4fce-befe-b36337e51a1d None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:50.354529 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:50.427806 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5bac14-09c1-4fce-befe-b36337e51a1d None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:50.460692 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c07c298-0fce-4293-ba4b-2a43e2cc3146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:50.461585 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07c298-0fce-4293-ba4b-2a43e2cc3146 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:50.461872 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07c298-0fce-4293-ba4b-2a43e2cc3146 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:50.462149 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07c298-0fce-4293-ba4b-2a43e2cc3146 None admin] PATH_INFO: `/v3/users/d14a75a485fd423bab1201b7d7847aa3` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:50.463340 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c07c298-0fce-4293-ba4b-2a43e2cc3146 None admin] RBAC: Authorizing `identity:get_user(user_id=d14a75a485fd423bab1201b7d7847aa3)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:50.464772 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c07c298-0fce-4293-ba4b-2a43e2cc3146 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:50.466482 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 609/1197] 10.222.0.36 () {56 vars in 1177 bytes} [Thu May 19 16:30:50 2022] GET /identity/v3/users/d14a75a485fd423bab1201b7d7847aa3 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:30:50.475187 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abef6d8-b4a0-4083-ab0d-4d4c32c4c298 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:50.521799 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:50.578207 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abef6d8-b4a0-4083-ab0d-4d4c32c4c298 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:50.612750 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c436ea3d-4a4c-4eab-bc00-e59e8a3a46f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:50.613368 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c436ea3d-4a4c-4eab-bc00-e59e8a3a46f7 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:50.613660 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c436ea3d-4a4c-4eab-bc00-e59e8a3a46f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:50.613944 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c436ea3d-4a4c-4eab-bc00-e59e8a3a46f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:50.680340 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c436ea3d-4a4c-4eab-bc00-e59e8a3a46f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:50.681832 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c436ea3d-4a4c-4eab-bc00-e59e8a3a46f7 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:50.725767 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:50.782863 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 589/1198] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:30:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 16:30:50.789010 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c07c298-0fce-4293-ba4b-2a43e2cc3146 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:50.881423 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:30:50.889069 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c07c298-0fce-4293-ba4b-2a43e2cc3146 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:50.925067 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45493bdf-517d-4f40-a0cd-1362189e40d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:50.925969 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45493bdf-517d-4f40-a0cd-1362189e40d7 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:50.926252 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45493bdf-517d-4f40-a0cd-1362189e40d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:50.926526 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45493bdf-517d-4f40-a0cd-1362189e40d7 None admin] PATH_INFO: `/v3/projects/35540a5a3ee84511b8d8cab14689a3b9` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:50.927731 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45493bdf-517d-4f40-a0cd-1362189e40d7 None admin] RBAC: Authorizing `identity:get_project(project_id=35540a5a3ee84511b8d8cab14689a3b9)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:50.929205 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45493bdf-517d-4f40-a0cd-1362189e40d7 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:50.930906 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 610/1199] 10.222.0.36 () {56 vars in 1186 bytes} [Thu May 19 16:30:50 2022] GET /identity/v3/projects/35540a5a3ee84511b8d8cab14689a3b9 => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:30:50.939165 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c436ea3d-4a4c-4eab-bc00-e59e8a3a46f7 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:30:51.039624 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c436ea3d-4a4c-4eab-bc00-e59e8a3a46f7 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:30:51.077170 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768cd9bc-8ad9-442a-a204-066c6b935b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:30:51.077817 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768cd9bc-8ad9-442a-a204-066c6b935b08 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:30:51.078102 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768cd9bc-8ad9-442a-a204-066c6b935b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:30:51.078381 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768cd9bc-8ad9-442a-a204-066c6b935b08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:30:51.079220 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768cd9bc-8ad9-442a-a204-066c6b935b08 None admin] RBAC: Adding query filter params (role.id=e2915fe109244a07a9be96bd2b219fdd, scope.project.id=35540a5a3ee84511b8d8cab14689a3b9, user.id=d14a75a485fd423bab1201b7d7847aa3) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:30:51.079538 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768cd9bc-8ad9-442a-a204-066c6b935b08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:30:51.080969 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768cd9bc-8ad9-442a-a204-066c6b935b08 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:30:51.086821 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 590/1200] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:30:50 2022] GET /identity/v3/role_assignments?user.id=d14a75a485fd423bab1201b7d7847aa3&scope.project.id=35540a5a3ee84511b8d8cab14689a3b9&role.id=e2915fe109244a07a9be96bd2b219fdd => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:31:00.920481 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=19 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:31:01.084536 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=95 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:31:52.552961 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab5f4c4-f183-4c88-b38b-539140c58593 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:31:52.552961 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab5f4c4-f183-4c88-b38b-539140c58593 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:31:52.552961 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab5f4c4-f183-4c88-b38b-539140c58593 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:31:52.554051 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 611/1201] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:31:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:31:52.560422 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069fd6e1-b51d-4d20-a542-1ea2e4e2b3e1 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:31:52.560970 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069fd6e1-b51d-4d20-a542-1ea2e4e2b3e1 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:31:52.560970 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069fd6e1-b51d-4d20-a542-1ea2e4e2b3e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:31:52.580599 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-069fd6e1-b51d-4d20-a542-1ea2e4e2b3e1 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:31:52.582265 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-069fd6e1-b51d-4d20-a542-1ea2e4e2b3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:31:52.691373 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 591/1202] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:31:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:31:52.711419 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab5f4c4-f183-4c88-b38b-539140c58593 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:31:52.852293 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab5f4c4-f183-4c88-b38b-539140c58593 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:31:52.895319 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73636e21-0a20-4295-ba64-00d16e16a8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:31:52.896374 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73636e21-0a20-4295-ba64-00d16e16a8c8 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:31:52.896697 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73636e21-0a20-4295-ba64-00d16e16a8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:31:52.896946 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73636e21-0a20-4295-ba64-00d16e16a8c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:31:52.909391 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73636e21-0a20-4295-ba64-00d16e16a8c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:31:52.910897 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73636e21-0a20-4295-ba64-00d16e16a8c8 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:31:52.937545 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-73636e21-0a20-4295-ba64-00d16e16a8c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:31:52.937545 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:31:52.937545 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:31:52.937545 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:31:52.937545 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:31:52.937545 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:31:52.937545 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:31:52.937545 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:31:52.937545 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:31:52.937545 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:31:52.937545 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:31:52.937545 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:31:52.937545 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:31:52.937545 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:31:52.937545 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:31:52.938967 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:31:52.938967 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:31:52.938967 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:31:52.938967 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:31:52.938967 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:31:52.938967 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:31:52.938967 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:31:52.938967 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:31:52.938967 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:31:52.938967 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:31:52.938967 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:31:52.938967 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:31:52.938967 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:31:52.938967 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:31:52.938967 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:31:52.938967 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:31:52.938967 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:31:52.940232 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:31:52.940232 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:31:52.940232 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:31:52.940232 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:31:52.942162 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 612/1203] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:31:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 16:31:52.949144 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069fd6e1-b51d-4d20-a542-1ea2e4e2b3e1 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:31:53.077907 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069fd6e1-b51d-4d20-a542-1ea2e4e2b3e1 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:31:53.118964 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:31:53.129953 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df2d437-21f0-42e1-89da-43cb62c9e5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:31:53.130846 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df2d437-21f0-42e1-89da-43cb62c9e5f6 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:31:53.131465 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df2d437-21f0-42e1-89da-43cb62c9e5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:31:53.131940 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df2d437-21f0-42e1-89da-43cb62c9e5f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:31:53.132942 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df2d437-21f0-42e1-89da-43cb62c9e5f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:31:53.133291 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df2d437-21f0-42e1-89da-43cb62c9e5f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:31:53.135341 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df2d437-21f0-42e1-89da-43cb62c9e5f6 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:31:53.150592 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 592/1204] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:31:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:31:53.156543 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73636e21-0a20-4295-ba64-00d16e16a8c8 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:31:53.230326 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:31:53.297747 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73636e21-0a20-4295-ba64-00d16e16a8c8 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:31:53.345511 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd950ba-ab15-445d-9d04-aa9a0b8acaae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:31:53.346564 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd950ba-ab15-445d-9d04-aa9a0b8acaae None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:31:53.347030 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd950ba-ab15-445d-9d04-aa9a0b8acaae None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:31:53.347510 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd950ba-ab15-445d-9d04-aa9a0b8acaae None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:31:53.348208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd950ba-ab15-445d-9d04-aa9a0b8acaae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:31:53.349741 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd950ba-ab15-445d-9d04-aa9a0b8acaae None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:31:53.385435 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 613/1205] 10.222.0.36 () {60 vars in 1132 bytes} [Thu May 19 16:31:53 2022] POST /identity/v3/users => generated 251 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:31:55.927480 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79559968-308d-4421-9cb2-fed97ffece46 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:31:55.929288 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79559968-308d-4421-9cb2-fed97ffece46 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:31:55.929288 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79559968-308d-4421-9cb2-fed97ffece46 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:31:55.929733 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 593/1206] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:31:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:31:55.935294 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6247a19-aa96-4470-979a-b93b4688d502 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:31:55.935750 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6247a19-aa96-4470-979a-b93b4688d502 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:31:55.935918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6247a19-aa96-4470-979a-b93b4688d502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:31:55.956248 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-f6247a19-aa96-4470-979a-b93b4688d502 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:31:55.957726 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-f6247a19-aa96-4470-979a-b93b4688d502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:31:56.037988 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 614/1207] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:31:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:31:56.061877 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79559968-308d-4421-9cb2-fed97ffece46 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:31:56.184283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79559968-308d-4421-9cb2-fed97ffece46 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:31:56.225063 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb696c82-e9d7-4c5a-a1fa-a23849ec1d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:31:56.225898 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb696c82-e9d7-4c5a-a1fa-a23849ec1d09 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:31:56.226185 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb696c82-e9d7-4c5a-a1fa-a23849ec1d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:31:56.226461 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb696c82-e9d7-4c5a-a1fa-a23849ec1d09 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:31:56.239917 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb696c82-e9d7-4c5a-a1fa-a23849ec1d09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:31:56.249508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-eb696c82-e9d7-4c5a-a1fa-a23849ec1d09 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 16:31:56.249508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:31:56.249508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:31:56.249508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:31:56.249508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:31:56.249508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:31:56.249508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:31:56.249508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:31:56.249508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:31:56.249508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:31:56.249508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:31:56.249508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:31:56.249508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:31:56.249508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:31:56.249508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:31:56.249508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:31:56.250595 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:31:56.250595 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:31:56.250595 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:31:56.250595 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:31:56.250595 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:31:56.250595 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:31:56.250595 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:31:56.250595 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:31:56.250595 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:31:56.250595 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:31:56.250595 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:31:56.250595 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:31:56.250595 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:31:56.250595 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:31:56.250595 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:31:56.250595 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:31:56.251535 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:31:56.251535 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:31:56.251535 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:31:56.251535 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:31:56.251535 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:31:56.251535 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:31:56.251535 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:31:56.251535 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:31:56.251535 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:31:56.251535 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:31:56.251535 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 16:31:56.251535 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:31:56.251535 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 594/1208] 10.222.0.36 () {56 vars in 1102 bytes} [Thu May 19 16:31:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:31:56.257778 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6247a19-aa96-4470-979a-b93b4688d502 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:31:56.315307 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:31:56.404392 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6247a19-aa96-4470-979a-b93b4688d502 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:31:56.443772 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b15a90b-275f-4249-9235-4e8351a3e14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:31:56.444409 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b15a90b-275f-4249-9235-4e8351a3e14e None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:31:56.444696 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b15a90b-275f-4249-9235-4e8351a3e14e None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:31:56.444999 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b15a90b-275f-4249-9235-4e8351a3e14e None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:31:56.445772 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b15a90b-275f-4249-9235-4e8351a3e14e None admin] RBAC: Adding query filter params (name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:31:56.446088 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b15a90b-275f-4249-9235-4e8351a3e14e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:31:56.448176 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b15a90b-275f-4249-9235-4e8351a3e14e None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:31:56.457651 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 615/1209] 10.222.0.36 () {56 vars in 1103 bytes} [Thu May 19 16:31:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:31:56.464467 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb696c82-e9d7-4c5a-a1fa-a23849ec1d09 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:31:56.532903 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:31:56.606586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb696c82-e9d7-4c5a-a1fa-a23849ec1d09 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:31:56.656556 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd54c4e5-d36f-44fe-8e27-2e338b4b9b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:31:56.657572 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54c4e5-d36f-44fe-8e27-2e338b4b9b33 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:31:56.657990 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54c4e5-d36f-44fe-8e27-2e338b4b9b33 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:31:56.658375 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54c4e5-d36f-44fe-8e27-2e338b4b9b33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:31:56.673798 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd54c4e5-d36f-44fe-8e27-2e338b4b9b33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:31:56.675457 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd54c4e5-d36f-44fe-8e27-2e338b4b9b33 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:31:56.698360 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-dd54c4e5-d36f-44fe-8e27-2e338b4b9b33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:31:56.698360 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:31:56.698360 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:31:56.698360 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:31:56.698360 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:31:56.698360 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:31:56.698360 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:31:56.698360 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:31:56.698360 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:31:56.698360 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:31:56.698360 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:31:56.698360 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:31:56.698360 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:31:56.698360 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:31:56.698360 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:31:56.699342 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:31:56.699342 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:31:56.699342 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:31:56.699342 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:31:56.699342 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:31:56.699342 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:31:56.699342 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:31:56.699342 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:31:56.699342 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:31:56.699342 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:31:56.699342 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:31:56.699342 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:31:56.699342 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:31:56.699342 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:31:56.699342 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:31:56.699342 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:31:56.699342 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:31:56.700497 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:31:56.700497 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:31:56.700497 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:31:56.700497 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:31:56.700497 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 595/1210] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:31:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:31:56.704781 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b15a90b-275f-4249-9235-4e8351a3e14e None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:31:56.825240 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:31:56.830768 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b15a90b-275f-4249-9235-4e8351a3e14e None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:31:56.870007 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a1dd2f-ff8b-46ba-8151-24a6e8c21364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:31:56.870734 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a1dd2f-ff8b-46ba-8151-24a6e8c21364 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:31:56.871023 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a1dd2f-ff8b-46ba-8151-24a6e8c21364 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:31:56.871283 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a1dd2f-ff8b-46ba-8151-24a6e8c21364 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:31:56.872017 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a1dd2f-ff8b-46ba-8151-24a6e8c21364 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:31:56.872333 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a1dd2f-ff8b-46ba-8151-24a6e8c21364 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:31:56.873847 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a1dd2f-ff8b-46ba-8151-24a6e8c21364 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:31:56.892193 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 616/1211] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:31:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:31:56.893334 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd54c4e5-d36f-44fe-8e27-2e338b4b9b33 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:31:57.004822 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd54c4e5-d36f-44fe-8e27-2e338b4b9b33 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:31:57.006547 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:31:57.050426 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544512ae-a1b2-4cdb-a94a-5093def94a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:31:57.051304 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544512ae-a1b2-4cdb-a94a-5093def94a83 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:31:57.051600 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544512ae-a1b2-4cdb-a94a-5093def94a83 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:31:57.051895 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544512ae-a1b2-4cdb-a94a-5093def94a83 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:31:57.061774 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544512ae-a1b2-4cdb-a94a-5093def94a83 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:31:57.063453 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544512ae-a1b2-4cdb-a94a-5093def94a83 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:31:57.080187 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-544512ae-a1b2-4cdb-a94a-5093def94a83 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 16:31:57.080187 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:31:57.080187 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:31:57.080187 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:31:57.080187 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:31:57.080187 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:31:57.080187 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:31:57.080187 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:31:57.080187 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:31:57.080187 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:31:57.080187 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:31:57.080187 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:31:57.080187 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 16:31:57.080187 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 16:31:57.080187 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 16:31:57.080187 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 16:31:57.081464 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:31:57.081464 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:31:57.081464 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 16:31:57.081464 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:31:57.081464 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 16:31:57.081464 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:31:57.081464 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:31:57.081464 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:31:57.081464 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:31:57.081464 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:31:57.081464 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:31:57.081464 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:31:57.081464 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:31:57.081464 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:31:57.081464 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:31:57.081464 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:31:57.081464 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:31:57.081464 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:31:57.082627 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:31:57.082627 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:31:57.082627 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 16:31:57.082627 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 16:31:57.082627 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 16:31:57.082627 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 16:31:57.082627 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 16:31:57.082627 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 16:31:57.082627 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 16:31:57.082627 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:31:57.082627 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 596/1212] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:31:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 16:31:57.087911 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a1dd2f-ff8b-46ba-8151-24a6e8c21364 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:31:57.218224 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a1dd2f-ff8b-46ba-8151-24a6e8c21364 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:31:57.262050 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f508dec6-cd7b-4da1-97f4-f8b538d8b10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:31:57.262712 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508dec6-cd7b-4da1-97f4-f8b538d8b10d None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:31:57.263004 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508dec6-cd7b-4da1-97f4-f8b538d8b10d None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:31:57.263277 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508dec6-cd7b-4da1-97f4-f8b538d8b10d None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:31:57.264208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f508dec6-cd7b-4da1-97f4-f8b538d8b10d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:31:57.264533 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f508dec6-cd7b-4da1-97f4-f8b538d8b10d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:31:57.266342 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f508dec6-cd7b-4da1-97f4-f8b538d8b10d None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:31:57.290663 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 617/1213] 10.222.0.36 () {56 vars in 1143 bytes} [Thu May 19 16:31:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:31:57.291873 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544512ae-a1b2-4cdb-a94a-5093def94a83 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:31:57.414227 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544512ae-a1b2-4cdb-a94a-5093def94a83 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:31:57.460674 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4163ae01-4300-45ff-a9bb-f3c6f20f08c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:31:57.461444 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163ae01-4300-45ff-a9bb-f3c6f20f08c4 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:31:57.461761 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163ae01-4300-45ff-a9bb-f3c6f20f08c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:31:57.462723 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163ae01-4300-45ff-a9bb-f3c6f20f08c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:31:57.494564 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:31:57.545671 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4163ae01-4300-45ff-a9bb-f3c6f20f08c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:31:57.547627 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4163ae01-4300-45ff-a9bb-f3c6f20f08c4 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:31:57.669036 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 597/1214] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:31:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 16:31:57.675213 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f508dec6-cd7b-4da1-97f4-f8b538d8b10d None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:31:57.696853 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:31:57.804296 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f508dec6-cd7b-4da1-97f4-f8b538d8b10d None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:31:57.847475 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2702d8-cb3c-4b80-9a56-a79ee7f06658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:31:57.848461 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2702d8-cb3c-4b80-9a56-a79ee7f06658 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:31:57.848750 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2702d8-cb3c-4b80-9a56-a79ee7f06658 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:31:57.849057 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2702d8-cb3c-4b80-9a56-a79ee7f06658 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:31:57.864047 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2702d8-cb3c-4b80-9a56-a79ee7f06658 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:31:57.865952 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2702d8-cb3c-4b80-9a56-a79ee7f06658 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:31:57.886658 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-6b2702d8-cb3c-4b80-9a56-a79ee7f06658 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:31:57.886658 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:31:57.886658 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:31:57.886658 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:31:57.886658 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:31:57.886658 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:31:57.886658 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:31:57.886658 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:31:57.886658 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:31:57.886658 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:31:57.886658 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:31:57.886658 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:31:57.886658 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:31:57.886658 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:31:57.886658 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:31:57.887754 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:31:57.887754 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:31:57.887754 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:31:57.887754 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:31:57.887754 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:31:57.887754 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:31:57.887754 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:31:57.887754 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:31:57.887754 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:31:57.887754 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:31:57.887754 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:31:57.887754 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:31:57.887754 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:31:57.887754 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:31:57.887754 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:31:57.887754 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:31:57.887754 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:31:57.888660 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:31:57.888660 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:31:57.888660 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:31:57.888660 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:31:57.894224 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4163ae01-4300-45ff-a9bb-f3c6f20f08c4 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:31:57.894540 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 618/1215] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:31:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 16:31:57.985924 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:31:58.035124 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4163ae01-4300-45ff-a9bb-f3c6f20f08c4 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:31:58.070657 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d9ea51-b58a-424c-984e-fbc9570e2f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:31:58.071379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d9ea51-b58a-424c-984e-fbc9570e2f26 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:31:58.071736 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d9ea51-b58a-424c-984e-fbc9570e2f26 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:31:58.072087 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d9ea51-b58a-424c-984e-fbc9570e2f26 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:31:58.072894 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d9ea51-b58a-424c-984e-fbc9570e2f26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:31:58.073315 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d9ea51-b58a-424c-984e-fbc9570e2f26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:31:58.074760 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d9ea51-b58a-424c-984e-fbc9570e2f26 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:31:58.087367 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 598/1216] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:31:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:31:58.093793 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2702d8-cb3c-4b80-9a56-a79ee7f06658 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:31:58.176267 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:31:58.212666 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2702d8-cb3c-4b80-9a56-a79ee7f06658 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:31:58.251992 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043c135c-2008-40c8-9983-771c33f3669d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:31:58.253119 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c135c-2008-40c8-9983-771c33f3669d None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:31:58.253536 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c135c-2008-40c8-9983-771c33f3669d None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:31:58.253928 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c135c-2008-40c8-9983-771c33f3669d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:31:58.262219 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043c135c-2008-40c8-9983-771c33f3669d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:31:58.263983 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043c135c-2008-40c8-9983-771c33f3669d None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:31:58.280765 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-043c135c-2008-40c8-9983-771c33f3669d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:31:58.280765 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:31:58.280765 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:31:58.280765 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:31:58.280765 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:31:58.280765 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:31:58.280765 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:31:58.280765 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:31:58.280765 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:31:58.280765 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:31:58.280765 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:31:58.280765 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:31:58.280765 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 16:31:58.280765 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 16:31:58.280765 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 16:31:58.282151 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 16:31:58.282151 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:31:58.282151 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:31:58.282151 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:31:58.282151 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:31:58.282151 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:31:58.282151 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:31:58.282151 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:31:58.282151 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:31:58.282151 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:31:58.282151 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:31:58.282151 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:31:58.282151 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:31:58.282151 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:31:58.282151 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:31:58.282151 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:31:58.282151 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:31:58.283146 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 16:31:58.283146 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 16:31:58.283146 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 16:31:58.283146 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 16:31:58.283146 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 16:31:58.283146 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 16:31:58.283146 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:31:58.283146 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:31:58.283146 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 619/1217] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:31:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:31:58.288276 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d9ea51-b58a-424c-984e-fbc9570e2f26 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:31:58.418808 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d9ea51-b58a-424c-984e-fbc9570e2f26 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:31:58.449304 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:31:58.464328 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fdc1d8-82ce-4f0a-92a7-ad7ecb9cf4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:31:58.465066 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fdc1d8-82ce-4f0a-92a7-ad7ecb9cf4ee None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:31:58.465422 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fdc1d8-82ce-4f0a-92a7-ad7ecb9cf4ee None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:31:58.465725 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fdc1d8-82ce-4f0a-92a7-ad7ecb9cf4ee None admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:31:58.466608 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fdc1d8-82ce-4f0a-92a7-ad7ecb9cf4ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:31:58.466983 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fdc1d8-82ce-4f0a-92a7-ad7ecb9cf4ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:31:58.468531 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fdc1d8-82ce-4f0a-92a7-ad7ecb9cf4ee None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:31:58.483738 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 599/1218] 10.222.0.36 () {56 vars in 1148 bytes} [Thu May 19 16:31:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:31:58.490574 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043c135c-2008-40c8-9983-771c33f3669d None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:31:58.611978 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043c135c-2008-40c8-9983-771c33f3669d None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:31:58.650134 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:31:58.660587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca19575b-e26f-4f72-aa1b-f5b063603490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:31:58.661307 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca19575b-e26f-4f72-aa1b-f5b063603490 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:31:58.661602 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca19575b-e26f-4f72-aa1b-f5b063603490 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:31:58.661888 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca19575b-e26f-4f72-aa1b-f5b063603490 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:31:58.663146 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca19575b-e26f-4f72-aa1b-f5b063603490 None admin] RBAC: Adding query filter params (role.id=727026b648e1429ca3697e04800b328a, scope.project.id=631c76794a124d6da22c626e590e4352, user.id=c6e158e5c87f4954aeb297f42dabdc16) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:31:58.663641 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca19575b-e26f-4f72-aa1b-f5b063603490 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:31:58.665379 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca19575b-e26f-4f72-aa1b-f5b063603490 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:31:58.673179 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 620/1219] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:31:58 2022] GET /identity/v3/role_assignments?user.id=c6e158e5c87f4954aeb297f42dabdc16&scope.project.id=631c76794a124d6da22c626e590e4352&role.id=727026b648e1429ca3697e04800b328a => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:32:01.232476 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780284b-19cd-448f-9fb8-67537b57ab1c None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:01.232990 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780284b-19cd-448f-9fb8-67537b57ab1c None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:01.232990 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780284b-19cd-448f-9fb8-67537b57ab1c None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:01.234397 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 600/1220] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:32:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:32:01.239590 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0373b707-b900-48f0-952f-922253ed189c None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:01.240012 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0373b707-b900-48f0-952f-922253ed189c None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:01.240158 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0373b707-b900-48f0-952f-922253ed189c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:01.262925 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-0373b707-b900-48f0-952f-922253ed189c None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:32:01.264965 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-0373b707-b900-48f0-952f-922253ed189c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:32:01.340804 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 621/1221] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:32:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:32:01.365777 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2780284b-19cd-448f-9fb8-67537b57ab1c None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:01.486255 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2780284b-19cd-448f-9fb8-67537b57ab1c None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:01.524426 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb23f0-73f9-4ab6-ac34-7310c884dc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:01.525288 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb23f0-73f9-4ab6-ac34-7310c884dc49 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:01.525579 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb23f0-73f9-4ab6-ac34-7310c884dc49 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:01.525862 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb23f0-73f9-4ab6-ac34-7310c884dc49 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:01.538577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bb23f0-73f9-4ab6-ac34-7310c884dc49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:32:01.547643 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-d3bb23f0-73f9-4ab6-ac34-7310c884dc49 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 16:32:01.547643 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:32:01.547643 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:32:01.547643 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:32:01.547643 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:32:01.547643 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:32:01.547643 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:32:01.547643 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:32:01.547643 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:32:01.547643 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:32:01.547643 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:32:01.547643 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:32:01.547643 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:32:01.547643 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:32:01.547643 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:32:01.547643 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:32:01.548672 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:32:01.548672 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:32:01.548672 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:32:01.548672 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:32:01.548672 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:32:01.548672 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:32:01.548672 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:32:01.548672 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:32:01.548672 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:32:01.548672 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:32:01.548672 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:32:01.548672 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:32:01.548672 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:32:01.548672 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:32:01.548672 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:32:01.548672 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:32:01.549798 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:32:01.549798 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:32:01.549798 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:32:01.549798 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:32:01.549798 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:32:01.549798 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:32:01.549798 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:32:01.549798 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:32:01.549798 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:32:01.549798 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:32:01.549798 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 16:32:01.549798 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:32:01.549798 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 601/1222] 10.222.0.36 () {56 vars in 1102 bytes} [Thu May 19 16:32:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:32:01.556117 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0373b707-b900-48f0-952f-922253ed189c None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:01.670753 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0373b707-b900-48f0-952f-922253ed189c None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:01.698841 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:01.712546 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c91f7-87e9-45d6-9f14-a0eb3650904b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:01.713498 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c91f7-87e9-45d6-9f14-a0eb3650904b None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:01.713918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c91f7-87e9-45d6-9f14-a0eb3650904b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:01.714204 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c91f7-87e9-45d6-9f14-a0eb3650904b None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:01.714972 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246c91f7-87e9-45d6-9f14-a0eb3650904b None admin] RBAC: Adding query filter params (name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:32:01.715286 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246c91f7-87e9-45d6-9f14-a0eb3650904b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:01.716685 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246c91f7-87e9-45d6-9f14-a0eb3650904b None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:01.727026 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 622/1223] 10.222.0.36 () {56 vars in 1103 bytes} [Thu May 19 16:32:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:32:01.734280 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb23f0-73f9-4ab6-ac34-7310c884dc49 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:01.791578 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:01.857666 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb23f0-73f9-4ab6-ac34-7310c884dc49 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:01.915670 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f26414-7568-4ae8-8f2e-d422d1698244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:01.916721 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f26414-7568-4ae8-8f2e-d422d1698244 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:01.917132 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f26414-7568-4ae8-8f2e-d422d1698244 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:01.917478 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f26414-7568-4ae8-8f2e-d422d1698244 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:01.931827 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f26414-7568-4ae8-8f2e-d422d1698244 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:01.933480 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f26414-7568-4ae8-8f2e-d422d1698244 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:01.955517 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-22f26414-7568-4ae8-8f2e-d422d1698244 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:32:01.955517 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:32:01.955517 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:32:01.955517 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:32:01.955517 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:32:01.955517 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:32:01.955517 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:32:01.955517 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:32:01.955517 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:32:01.955517 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:32:01.955517 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:32:01.955517 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:32:01.955517 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:32:01.955517 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:32:01.955517 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:32:01.956667 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:32:01.956667 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:32:01.956667 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:32:01.956667 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:32:01.956667 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:32:01.956667 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:32:01.956667 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:32:01.956667 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:32:01.956667 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:32:01.956667 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:32:01.956667 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:32:01.956667 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:32:01.956667 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:32:01.956667 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:32:01.956667 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:32:01.956667 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:32:01.956667 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:32:01.957637 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:32:01.957637 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:32:01.957637 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:32:01.957637 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:32:01.957637 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 602/1224] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:32:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:32:01.963158 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c91f7-87e9-45d6-9f14-a0eb3650904b None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:02.082294 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c91f7-87e9-45d6-9f14-a0eb3650904b None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:02.122866 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039a5e16-42eb-4964-8b95-f6affe357d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:02.123682 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a5e16-42eb-4964-8b95-f6affe357d48 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:02.123968 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a5e16-42eb-4964-8b95-f6affe357d48 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:02.124250 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a5e16-42eb-4964-8b95-f6affe357d48 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:02.125061 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039a5e16-42eb-4964-8b95-f6affe357d48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:32:02.125415 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039a5e16-42eb-4964-8b95-f6affe357d48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:02.126799 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039a5e16-42eb-4964-8b95-f6affe357d48 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:02.141202 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 623/1225] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:32:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:32:02.147628 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f26414-7568-4ae8-8f2e-d422d1698244 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:02.253221 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:02.262733 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f26414-7568-4ae8-8f2e-d422d1698244 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:02.301298 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eebb0d-619f-4930-9c2e-15ed95861d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:02.302113 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eebb0d-619f-4930-9c2e-15ed95861d09 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:02.302399 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eebb0d-619f-4930-9c2e-15ed95861d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:02.302680 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eebb0d-619f-4930-9c2e-15ed95861d09 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:02.310440 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eebb0d-619f-4930-9c2e-15ed95861d09 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:02.311883 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eebb0d-619f-4930-9c2e-15ed95861d09 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:02.326657 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-e5eebb0d-619f-4930-9c2e-15ed95861d09 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 16:32:02.326657 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:32:02.326657 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:32:02.326657 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:32:02.326657 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:32:02.326657 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:32:02.326657 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:32:02.326657 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:32:02.326657 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:32:02.326657 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:32:02.326657 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:32:02.326657 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:32:02.326657 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 16:32:02.326657 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 16:32:02.326657 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 16:32:02.326657 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 16:32:02.327845 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:32:02.327845 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:32:02.327845 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 16:32:02.327845 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:32:02.327845 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 16:32:02.327845 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:32:02.327845 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:32:02.327845 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:32:02.327845 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:32:02.327845 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:32:02.327845 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:32:02.327845 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:32:02.327845 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:32:02.327845 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:32:02.327845 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:32:02.327845 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:32:02.327845 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:32:02.327845 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:32:02.329020 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:32:02.329020 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:32:02.329020 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 16:32:02.329020 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 16:32:02.329020 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 16:32:02.329020 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 16:32:02.329020 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 16:32:02.329020 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 16:32:02.329020 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 16:32:02.329020 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:32:02.329020 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 603/1226] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:32:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:32:02.336602 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039a5e16-42eb-4964-8b95-f6affe357d48 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:02.388023 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:02.451830 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039a5e16-42eb-4964-8b95-f6affe357d48 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:02.489842 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e2b1de-c221-49c3-bf52-a3701a8067cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:02.490700 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e2b1de-c221-49c3-bf52-a3701a8067cf None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:02.491104 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e2b1de-c221-49c3-bf52-a3701a8067cf None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:02.491554 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e2b1de-c221-49c3-bf52-a3701a8067cf None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:02.492502 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e2b1de-c221-49c3-bf52-a3701a8067cf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:32:02.492928 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e2b1de-c221-49c3-bf52-a3701a8067cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:02.494984 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e2b1de-c221-49c3-bf52-a3701a8067cf None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:02.513754 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 624/1227] 10.222.0.36 () {56 vars in 1143 bytes} [Thu May 19 16:32:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:32:02.519332 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eebb0d-619f-4930-9c2e-15ed95861d09 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:02.640642 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eebb0d-619f-4930-9c2e-15ed95861d09 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:02.693147 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe808c5d-bbc7-4c1f-978f-fa6c60b25057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:02.694189 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe808c5d-bbc7-4c1f-978f-fa6c60b25057 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:02.694370 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe808c5d-bbc7-4c1f-978f-fa6c60b25057 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:02.694650 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe808c5d-bbc7-4c1f-978f-fa6c60b25057 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:02.708897 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe808c5d-bbc7-4c1f-978f-fa6c60b25057 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:02.710885 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe808c5d-bbc7-4c1f-978f-fa6c60b25057 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:02.731475 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-fe808c5d-bbc7-4c1f-978f-fa6c60b25057 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:32:02.731475 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:32:02.731475 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:32:02.731475 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:32:02.731475 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:32:02.731475 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:32:02.731475 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:32:02.731475 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:32:02.731475 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:32:02.731475 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:32:02.731475 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:32:02.731475 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:32:02.731475 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:32:02.731475 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:32:02.731475 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:32:02.732591 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:32:02.732591 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:32:02.732591 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:32:02.732591 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:32:02.732591 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:32:02.732591 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:32:02.732591 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:32:02.732591 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:32:02.732591 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:32:02.732591 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:32:02.732591 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:32:02.732591 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:32:02.732591 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:32:02.732591 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:32:02.732591 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:32:02.732591 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:32:02.732591 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:32:02.733697 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:32:02.733697 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:32:02.733697 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:32:02.733697 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:32:02.735304 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 604/1228] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:32:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:32:02.741554 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e2b1de-c221-49c3-bf52-a3701a8067cf None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:02.867170 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e2b1de-c221-49c3-bf52-a3701a8067cf None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:02.869762 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:02.905849 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de53490-28d1-46e4-a4fc-cbc56ade8025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:02.906525 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de53490-28d1-46e4-a4fc-cbc56ade8025 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:02.906812 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de53490-28d1-46e4-a4fc-cbc56ade8025 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:02.907089 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de53490-28d1-46e4-a4fc-cbc56ade8025 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:02.907815 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de53490-28d1-46e4-a4fc-cbc56ade8025 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:32:02.908140 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de53490-28d1-46e4-a4fc-cbc56ade8025 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:02.909582 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de53490-28d1-46e4-a4fc-cbc56ade8025 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:02.924296 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 625/1229] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:32:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:32:02.928015 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe808c5d-bbc7-4c1f-978f-fa6c60b25057 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:02.938601 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:03.041343 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe808c5d-bbc7-4c1f-978f-fa6c60b25057 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:03.076339 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6c4c4d-7f7b-4811-b399-e77af7054b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:03.077182 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c4c4d-7f7b-4811-b399-e77af7054b01 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:03.077461 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c4c4d-7f7b-4811-b399-e77af7054b01 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:03.077741 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c4c4d-7f7b-4811-b399-e77af7054b01 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:03.085220 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6c4c4d-7f7b-4811-b399-e77af7054b01 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:03.086783 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6c4c4d-7f7b-4811-b399-e77af7054b01 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:03.105111 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-9c6c4c4d-7f7b-4811-b399-e77af7054b01 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:32:03.105111 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:32:03.105111 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:32:03.105111 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:32:03.105111 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:32:03.105111 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:32:03.105111 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:32:03.105111 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:32:03.105111 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:32:03.105111 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:32:03.105111 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:32:03.105111 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:32:03.105111 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 16:32:03.105111 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 16:32:03.105111 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 16:32:03.106293 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 16:32:03.106293 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:32:03.106293 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:32:03.106293 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:32:03.106293 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:32:03.106293 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:32:03.106293 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:32:03.106293 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:32:03.106293 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:32:03.106293 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:32:03.106293 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:32:03.106293 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:32:03.106293 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:32:03.106293 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:32:03.106293 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:32:03.106293 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:32:03.106293 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:32:03.107491 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 16:32:03.107491 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 16:32:03.107491 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 16:32:03.107491 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 16:32:03.107491 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 16:32:03.107491 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 16:32:03.107491 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:32:03.107491 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:32:03.107491 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 605/1230] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:32:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:32:03.113014 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de53490-28d1-46e4-a4fc-cbc56ade8025 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:03.235203 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de53490-28d1-46e4-a4fc-cbc56ade8025 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:03.276902 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bd8d3c-2b8d-4680-a217-3f6148b7808d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:03.277782 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd8d3c-2b8d-4680-a217-3f6148b7808d None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:03.278161 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd8d3c-2b8d-4680-a217-3f6148b7808d None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:03.278447 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd8d3c-2b8d-4680-a217-3f6148b7808d None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:03.279360 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bd8d3c-2b8d-4680-a217-3f6148b7808d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:32:03.279828 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bd8d3c-2b8d-4680-a217-3f6148b7808d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:03.281388 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bd8d3c-2b8d-4680-a217-3f6148b7808d None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:03.297913 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 626/1231] 10.222.0.36 () {56 vars in 1148 bytes} [Thu May 19 16:32:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:32:03.304927 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6c4c4d-7f7b-4811-b399-e77af7054b01 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:03.379187 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:03.418147 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6c4c4d-7f7b-4811-b399-e77af7054b01 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:03.452905 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4769ff-4bad-4eb0-ae61-abcb50407b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:03.453882 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4769ff-4bad-4eb0-ae61-abcb50407b56 None admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:03.454172 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4769ff-4bad-4eb0-ae61-abcb50407b56 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:03.454451 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4769ff-4bad-4eb0-ae61-abcb50407b56 None admin] PATH_INFO: `/v3/projects/631c76794a124d6da22c626e590e4352/users/c6e158e5c87f4954aeb297f42dabdc16/roles/727026b648e1429ca3697e04800b328a` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:03.473191 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4769ff-4bad-4eb0-ae61-abcb50407b56 None admin] RBAC: Authorizing `identity:create_grant(project_id=631c76794a124d6da22c626e590e4352, user_id=c6e158e5c87f4954aeb297f42dabdc16, role_id=727026b648e1429ca3697e04800b328a)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:03.475521 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4769ff-4bad-4eb0-ae61-abcb50407b56 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:03.490774 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 606/1232] 10.222.0.36 () {58 vars in 1439 bytes} [Thu May 19 16:32:03 2022] PUT /identity/v3/projects/631c76794a124d6da22c626e590e4352/users/c6e158e5c87f4954aeb297f42dabdc16/roles/727026b648e1429ca3697e04800b328a => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:32:05.931031 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7854b7a-b8ae-4277-9aac-e2f52e30873a None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:05.931574 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7854b7a-b8ae-4277-9aac-e2f52e30873a None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:05.931687 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7854b7a-b8ae-4277-9aac-e2f52e30873a None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:05.933353 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 627/1233] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:32:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:32:05.940369 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97120668-1624-4670-9ae9-c716dd54d3c0 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:05.940889 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97120668-1624-4670-9ae9-c716dd54d3c0 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:05.941022 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97120668-1624-4670-9ae9-c716dd54d3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:05.961238 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-97120668-1624-4670-9ae9-c716dd54d3c0 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:32:05.963360 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-97120668-1624-4670-9ae9-c716dd54d3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:32:06.042936 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 607/1234] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:32:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:32:06.067575 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7854b7a-b8ae-4277-9aac-e2f52e30873a None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:06.094585 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:06.189007 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7854b7a-b8ae-4277-9aac-e2f52e30873a None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:06.226631 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd71153-9dd4-45ab-ad1b-dc9889b14ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:06.227456 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd71153-9dd4-45ab-ad1b-dc9889b14ad5 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:06.227812 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd71153-9dd4-45ab-ad1b-dc9889b14ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:06.228044 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd71153-9dd4-45ab-ad1b-dc9889b14ad5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:06.240728 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd71153-9dd4-45ab-ad1b-dc9889b14ad5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:32:06.250806 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-ffd71153-9dd4-45ab-ad1b-dc9889b14ad5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 16:32:06.250806 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:32:06.250806 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:32:06.250806 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:32:06.250806 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:32:06.250806 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:32:06.250806 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:32:06.250806 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:32:06.250806 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:32:06.250806 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:32:06.250806 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:32:06.250806 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:32:06.250806 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:32:06.250806 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:32:06.250806 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:32:06.250806 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:32:06.252219 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:32:06.252219 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:32:06.252219 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:32:06.252219 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:32:06.252219 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:32:06.252219 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:32:06.252219 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:32:06.252219 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:32:06.252219 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:32:06.252219 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:32:06.252219 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:32:06.252219 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:32:06.252219 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:32:06.252219 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:32:06.252219 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:32:06.252219 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:32:06.253101 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:32:06.253101 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:32:06.253101 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:32:06.253101 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:32:06.253101 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:32:06.253101 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:32:06.253101 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:32:06.253101 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:32:06.253101 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:32:06.253101 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:32:06.253101 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 16:32:06.253101 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:32:06.253101 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 628/1235] 10.222.0.36 () {56 vars in 1102 bytes} [Thu May 19 16:32:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:32:06.259057 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97120668-1624-4670-9ae9-c716dd54d3c0 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:06.310642 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:06.376433 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97120668-1624-4670-9ae9-c716dd54d3c0 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:06.417796 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c00af2-fae8-4e4c-b6ec-64316f98c2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:06.418581 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c00af2-fae8-4e4c-b6ec-64316f98c2bd None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:06.418877 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c00af2-fae8-4e4c-b6ec-64316f98c2bd None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:06.419197 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c00af2-fae8-4e4c-b6ec-64316f98c2bd None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:06.420142 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c00af2-fae8-4e4c-b6ec-64316f98c2bd None admin] RBAC: Adding query filter params (name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:32:06.420533 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c00af2-fae8-4e4c-b6ec-64316f98c2bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:06.422287 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c00af2-fae8-4e4c-b6ec-64316f98c2bd None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:06.433539 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 608/1236] 10.222.0.36 () {56 vars in 1103 bytes} [Thu May 19 16:32:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:32:06.440338 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd71153-9dd4-45ab-ad1b-dc9889b14ad5 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:06.542683 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:06.560714 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd71153-9dd4-45ab-ad1b-dc9889b14ad5 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:06.599263 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13783db-c372-4de9-bb64-069ac337838f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:06.600183 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13783db-c372-4de9-bb64-069ac337838f None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:06.600465 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13783db-c372-4de9-bb64-069ac337838f None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:06.600736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13783db-c372-4de9-bb64-069ac337838f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:06.612153 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13783db-c372-4de9-bb64-069ac337838f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:06.613665 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13783db-c372-4de9-bb64-069ac337838f None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:06.632493 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-d13783db-c372-4de9-bb64-069ac337838f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:32:06.632493 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:32:06.632493 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:32:06.632493 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:32:06.632493 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:32:06.632493 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:32:06.632493 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:32:06.632493 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:32:06.632493 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:32:06.632493 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:32:06.632493 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:32:06.632493 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:32:06.632493 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:32:06.632493 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:32:06.632493 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:32:06.634190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:32:06.634190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:32:06.634190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:32:06.634190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:32:06.634190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:32:06.634190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:32:06.634190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:32:06.634190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:32:06.634190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:32:06.634190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:32:06.634190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:32:06.634190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:32:06.634190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:32:06.634190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:32:06.634190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:32:06.634190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:32:06.634190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:32:06.635553 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:32:06.635553 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:32:06.635553 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:32:06.635553 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:32:06.635553 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 629/1237] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:32:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:32:06.640743 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c00af2-fae8-4e4c-b6ec-64316f98c2bd None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:06.764212 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c00af2-fae8-4e4c-b6ec-64316f98c2bd None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:06.774055 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:06.800438 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a993040-cd50-43f0-a057-c77d8b02f738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:06.801111 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a993040-cd50-43f0-a057-c77d8b02f738 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:06.801451 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a993040-cd50-43f0-a057-c77d8b02f738 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:06.801735 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a993040-cd50-43f0-a057-c77d8b02f738 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:06.802528 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a993040-cd50-43f0-a057-c77d8b02f738 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:32:06.802884 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a993040-cd50-43f0-a057-c77d8b02f738 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:06.804316 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a993040-cd50-43f0-a057-c77d8b02f738 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:06.817057 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 609/1238] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:32:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:32:06.823572 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13783db-c372-4de9-bb64-069ac337838f None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:06.941995 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13783db-c372-4de9-bb64-069ac337838f None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:06.978073 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6e8e3e-d359-4751-b5ab-c05b2b3e1461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:06.978908 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6e8e3e-d359-4751-b5ab-c05b2b3e1461 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:06.979182 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6e8e3e-d359-4751-b5ab-c05b2b3e1461 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:06.979468 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6e8e3e-d359-4751-b5ab-c05b2b3e1461 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:06.982347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:06.987572 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6e8e3e-d359-4751-b5ab-c05b2b3e1461 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:06.989255 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6e8e3e-d359-4751-b5ab-c05b2b3e1461 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:07.004050 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-1a6e8e3e-d359-4751-b5ab-c05b2b3e1461 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 16:32:07.004050 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:32:07.004050 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:32:07.004050 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:32:07.004050 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:32:07.004050 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:32:07.004050 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:32:07.004050 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:32:07.004050 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:32:07.004050 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:32:07.004050 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:32:07.004050 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:32:07.004050 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 16:32:07.004050 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 16:32:07.004050 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 16:32:07.004050 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 16:32:07.005076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:32:07.005076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:32:07.005076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 16:32:07.005076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:32:07.005076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 16:32:07.005076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:32:07.005076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:32:07.005076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:32:07.005076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:32:07.005076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:32:07.005076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:32:07.005076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:32:07.005076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:32:07.005076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:32:07.005076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:32:07.005076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:32:07.005076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:32:07.005076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:32:07.006087 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:32:07.006087 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:32:07.006087 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 16:32:07.006087 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 16:32:07.006087 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 16:32:07.006087 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 16:32:07.006087 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 16:32:07.006087 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 16:32:07.006087 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 16:32:07.006087 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:32:07.006087 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 630/1239] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:32:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 16:32:07.012187 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a993040-cd50-43f0-a057-c77d8b02f738 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:07.125642 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a993040-cd50-43f0-a057-c77d8b02f738 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:07.164496 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df8de5-0abd-4bd1-858e-243a9a535e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:07.165231 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df8de5-0abd-4bd1-858e-243a9a535e85 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:07.165523 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df8de5-0abd-4bd1-858e-243a9a535e85 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:07.165810 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df8de5-0abd-4bd1-858e-243a9a535e85 None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:07.166664 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59df8de5-0abd-4bd1-858e-243a9a535e85 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:32:07.166978 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59df8de5-0abd-4bd1-858e-243a9a535e85 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:07.169819 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59df8de5-0abd-4bd1-858e-243a9a535e85 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:07.198961 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 610/1240] 10.222.0.36 () {56 vars in 1143 bytes} [Thu May 19 16:32:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:32:07.204831 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6e8e3e-d359-4751-b5ab-c05b2b3e1461 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:07.324548 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6e8e3e-d359-4751-b5ab-c05b2b3e1461 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:07.366798 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e288e-ab36-462a-b1fe-1053eaa3c488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:07.368500 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e288e-ab36-462a-b1fe-1053eaa3c488 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:07.368895 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e288e-ab36-462a-b1fe-1053eaa3c488 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:07.369453 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e288e-ab36-462a-b1fe-1053eaa3c488 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:07.447783 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:07.455415 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e288e-ab36-462a-b1fe-1053eaa3c488 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:07.456909 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e288e-ab36-462a-b1fe-1053eaa3c488 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:07.570312 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 631/1241] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:32:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 16:32:07.576840 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df8de5-0abd-4bd1-858e-243a9a535e85 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:07.607049 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:07.695457 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df8de5-0abd-4bd1-858e-243a9a535e85 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:07.736709 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695ec28e-3a73-4f45-8663-404d4eaaacac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:07.737466 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ec28e-3a73-4f45-8663-404d4eaaacac None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:07.737760 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ec28e-3a73-4f45-8663-404d4eaaacac None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:07.738041 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ec28e-3a73-4f45-8663-404d4eaaacac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:07.749120 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695ec28e-3a73-4f45-8663-404d4eaaacac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:07.750697 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695ec28e-3a73-4f45-8663-404d4eaaacac None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:07.775080 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-695ec28e-3a73-4f45-8663-404d4eaaacac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:32:07.775080 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:32:07.775080 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:32:07.775080 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:32:07.775080 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:32:07.775080 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:32:07.775080 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:32:07.775080 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:32:07.775080 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:32:07.775080 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:32:07.775080 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:32:07.775080 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:32:07.775080 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:32:07.775080 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:32:07.775080 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:32:07.785586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:32:07.785586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:32:07.785586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:32:07.785586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:32:07.785586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:32:07.785586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:32:07.785586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:32:07.785586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:32:07.785586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:32:07.785586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:32:07.785586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:32:07.785586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:32:07.785586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:32:07.785586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:32:07.785586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:32:07.785586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:32:07.785586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:32:07.786721 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:32:07.786721 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:32:07.786721 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:32:07.786721 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:32:07.786721 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 611/1242] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:32:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 16:32:07.788165 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e288e-ab36-462a-b1fe-1053eaa3c488 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:07.898424 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:07.905526 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e288e-ab36-462a-b1fe-1053eaa3c488 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:07.944700 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d560d5e4-21b9-4e7a-9a35-dca9061ee1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:07.945424 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d560d5e4-21b9-4e7a-9a35-dca9061ee1c5 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:07.945670 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d560d5e4-21b9-4e7a-9a35-dca9061ee1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:07.946001 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d560d5e4-21b9-4e7a-9a35-dca9061ee1c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:07.946774 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d560d5e4-21b9-4e7a-9a35-dca9061ee1c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:32:07.947089 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d560d5e4-21b9-4e7a-9a35-dca9061ee1c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:07.948507 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d560d5e4-21b9-4e7a-9a35-dca9061ee1c5 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:07.960475 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 632/1243] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:32:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:32:07.967126 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695ec28e-3a73-4f45-8663-404d4eaaacac None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:08.060651 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:08.084656 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695ec28e-3a73-4f45-8663-404d4eaaacac None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:08.126840 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062381ca-9ad0-4420-962f-5188f4db876e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:08.127992 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062381ca-9ad0-4420-962f-5188f4db876e None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:08.128255 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062381ca-9ad0-4420-962f-5188f4db876e None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:08.128749 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062381ca-9ad0-4420-962f-5188f4db876e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:08.137711 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062381ca-9ad0-4420-962f-5188f4db876e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:08.139185 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062381ca-9ad0-4420-962f-5188f4db876e None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:08.158523 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-062381ca-9ad0-4420-962f-5188f4db876e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:32:08.158523 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:32:08.158523 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:32:08.158523 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:32:08.158523 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:32:08.158523 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:32:08.158523 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:32:08.158523 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:32:08.158523 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:32:08.158523 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:32:08.158523 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:32:08.158523 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:32:08.158523 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 16:32:08.158523 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 16:32:08.158523 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 16:32:08.159644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 16:32:08.159644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:32:08.159644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:32:08.159644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:32:08.159644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:32:08.159644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:32:08.159644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:32:08.159644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:32:08.159644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:32:08.159644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:32:08.159644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:32:08.159644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:32:08.159644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:32:08.159644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:32:08.159644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:32:08.159644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:32:08.159644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:32:08.160645 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 16:32:08.160645 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 16:32:08.160645 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 16:32:08.160645 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 16:32:08.160645 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 16:32:08.160645 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 16:32:08.160645 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:32:08.160645 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:32:08.160645 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 612/1244] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:32:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:32:08.166522 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d560d5e4-21b9-4e7a-9a35-dca9061ee1c5 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:08.294024 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d560d5e4-21b9-4e7a-9a35-dca9061ee1c5 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:08.337975 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b4447-9c40-4b28-8010-f360648c1373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:08.338706 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b4447-9c40-4b28-8010-f360648c1373 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:08.338996 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b4447-9c40-4b28-8010-f360648c1373 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:08.339434 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b4447-9c40-4b28-8010-f360648c1373 None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:08.340237 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b4447-9c40-4b28-8010-f360648c1373 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:32:08.340689 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b4447-9c40-4b28-8010-f360648c1373 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:08.342278 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b4447-9c40-4b28-8010-f360648c1373 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:08.357184 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 633/1245] 10.222.0.36 () {56 vars in 1148 bytes} [Thu May 19 16:32:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:32:08.363621 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062381ca-9ad0-4420-962f-5188f4db876e None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:08.492053 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062381ca-9ad0-4420-962f-5188f4db876e None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:08.529018 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6037cad7-b639-4150-a555-812a1f2708ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:08.529729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6037cad7-b639-4150-a555-812a1f2708ce None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:08.530020 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6037cad7-b639-4150-a555-812a1f2708ce None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:08.530310 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6037cad7-b639-4150-a555-812a1f2708ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:08.531168 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6037cad7-b639-4150-a555-812a1f2708ce None admin] RBAC: Adding query filter params (role.id=727026b648e1429ca3697e04800b328a, scope.project.id=631c76794a124d6da22c626e590e4352, user.id=c6e158e5c87f4954aeb297f42dabdc16) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:32:08.531615 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6037cad7-b639-4150-a555-812a1f2708ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:08.532990 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6037cad7-b639-4150-a555-812a1f2708ce None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:08.547333 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 613/1246] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:32:08 2022] GET /identity/v3/role_assignments?user.id=c6e158e5c87f4954aeb297f42dabdc16&scope.project.id=631c76794a124d6da22c626e590e4352&role.id=727026b648e1429ca3697e04800b328a => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:32:11.000798 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87880a6-7a40-4674-a5a7-92cdcc36cd9d None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:11.001588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87880a6-7a40-4674-a5a7-92cdcc36cd9d None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:11.001588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87880a6-7a40-4674-a5a7-92cdcc36cd9d None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:11.002880 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 634/1247] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:32:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:32:11.008926 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd26fe48-0a1c-4824-bdc0-33063729a096 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:11.009435 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd26fe48-0a1c-4824-bdc0-33063729a096 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:11.009551 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd26fe48-0a1c-4824-bdc0-33063729a096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:11.015000 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:11.031045 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-bd26fe48-0a1c-4824-bdc0-33063729a096 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:32:11.032327 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-bd26fe48-0a1c-4824-bdc0-33063729a096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:32:11.117992 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 614/1248] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:32:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:32:11.146924 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87880a6-7a40-4674-a5a7-92cdcc36cd9d None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:11.167697 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:11.277146 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87880a6-7a40-4674-a5a7-92cdcc36cd9d None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:11.322054 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5107aa0-add8-4f3a-9b7f-a4f62c2efc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:11.322909 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5107aa0-add8-4f3a-9b7f-a4f62c2efc25 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:11.323235 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5107aa0-add8-4f3a-9b7f-a4f62c2efc25 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:11.323592 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5107aa0-add8-4f3a-9b7f-a4f62c2efc25 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:11.339262 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5107aa0-add8-4f3a-9b7f-a4f62c2efc25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:32:11.349863 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-c5107aa0-add8-4f3a-9b7f-a4f62c2efc25 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:32:11.349863 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:32:11.349863 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:32:11.349863 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:32:11.349863 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:32:11.349863 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:32:11.349863 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:32:11.349863 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:32:11.349863 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:32:11.349863 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:32:11.349863 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:32:11.349863 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:32:11.349863 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:32:11.349863 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:32:11.349863 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:32:11.349863 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:32:11.351778 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:32:11.351778 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:32:11.351778 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:32:11.351778 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:32:11.351778 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:32:11.351778 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:32:11.351778 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:32:11.351778 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:32:11.351778 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:32:11.351778 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:32:11.351778 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:32:11.351778 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:32:11.351778 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:32:11.351778 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:32:11.351778 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:32:11.351778 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:32:11.353422 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:32:11.353422 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:32:11.353422 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:32:11.353422 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:32:11.353422 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:32:11.353422 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:32:11.353422 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:32:11.353422 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:32:11.353422 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:32:11.353422 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:32:11.353422 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:32:11.353422 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:32:11.357955 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 635/1249] 10.222.0.36 () {56 vars in 1096 bytes} [Thu May 19 16:32:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:32:11.359315 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd26fe48-0a1c-4824-bdc0-33063729a096 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:11.489201 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd26fe48-0a1c-4824-bdc0-33063729a096 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:11.519766 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:11.525893 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc388f12-5ec5-44db-845b-4fb6af264a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:11.526583 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc388f12-5ec5-44db-845b-4fb6af264a3e None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:11.526874 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc388f12-5ec5-44db-845b-4fb6af264a3e None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:11.527157 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc388f12-5ec5-44db-845b-4fb6af264a3e None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:11.527929 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc388f12-5ec5-44db-845b-4fb6af264a3e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:32:11.528243 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc388f12-5ec5-44db-845b-4fb6af264a3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:11.530047 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc388f12-5ec5-44db-845b-4fb6af264a3e None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:11.542300 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 615/1250] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:32:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:32:11.549420 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5107aa0-add8-4f3a-9b7f-a4f62c2efc25 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:11.642867 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:11.679807 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5107aa0-add8-4f3a-9b7f-a4f62c2efc25 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:11.726234 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e532e03-ccad-4272-9cf4-2ed4c345e761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:11.727048 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e532e03-ccad-4272-9cf4-2ed4c345e761 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:11.727331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e532e03-ccad-4272-9cf4-2ed4c345e761 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:11.727606 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e532e03-ccad-4272-9cf4-2ed4c345e761 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:11.744454 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e532e03-ccad-4272-9cf4-2ed4c345e761 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:11.746213 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e532e03-ccad-4272-9cf4-2ed4c345e761 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:11.767038 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-3e532e03-ccad-4272-9cf4-2ed4c345e761 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:32:11.767038 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:32:11.767038 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:32:11.767038 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:32:11.767038 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:32:11.767038 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:32:11.767038 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:32:11.767038 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:32:11.767038 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:32:11.767038 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:32:11.767038 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:32:11.767038 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:32:11.767038 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:32:11.767038 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:32:11.767038 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:32:11.768056 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:32:11.768056 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:32:11.768056 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:32:11.768056 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:32:11.768056 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:32:11.768056 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:32:11.768056 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:32:11.768056 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:32:11.768056 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:32:11.768056 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:32:11.768056 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:32:11.768056 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:32:11.768056 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:32:11.768056 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:32:11.768056 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:32:11.768056 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:32:11.768056 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:32:11.768960 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:32:11.768960 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:32:11.768960 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:32:11.768960 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:32:11.768960 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 636/1251] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:32:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 16:32:11.776757 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc388f12-5ec5-44db-845b-4fb6af264a3e None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:11.891239 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc388f12-5ec5-44db-845b-4fb6af264a3e None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:11.944517 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbb83fb-390c-4381-8282-6f73674d3c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:11.945215 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb83fb-390c-4381-8282-6f73674d3c51 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:11.945501 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb83fb-390c-4381-8282-6f73674d3c51 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:11.945870 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb83fb-390c-4381-8282-6f73674d3c51 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:11.946612 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbb83fb-390c-4381-8282-6f73674d3c51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:32:11.946924 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbb83fb-390c-4381-8282-6f73674d3c51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:11.948300 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbb83fb-390c-4381-8282-6f73674d3c51 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:11.964742 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 616/1252] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:32:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:32:11.972672 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e532e03-ccad-4272-9cf4-2ed4c345e761 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:12.106136 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e532e03-ccad-4272-9cf4-2ed4c345e761 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:12.135637 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:12.144570 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15803e1d-4d35-46d8-8352-fa86a8b6bd36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:12.145438 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15803e1d-4d35-46d8-8352-fa86a8b6bd36 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:12.145724 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15803e1d-4d35-46d8-8352-fa86a8b6bd36 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:12.146002 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15803e1d-4d35-46d8-8352-fa86a8b6bd36 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:12.154699 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15803e1d-4d35-46d8-8352-fa86a8b6bd36 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:12.156181 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15803e1d-4d35-46d8-8352-fa86a8b6bd36 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:12.172214 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-15803e1d-4d35-46d8-8352-fa86a8b6bd36 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 16:32:12.172214 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:32:12.172214 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:32:12.172214 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:32:12.172214 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:32:12.172214 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:32:12.172214 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:32:12.172214 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:32:12.172214 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:32:12.172214 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:32:12.172214 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:32:12.172214 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:32:12.172214 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 16:32:12.172214 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 16:32:12.172214 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 16:32:12.172214 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 16:32:12.173470 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:32:12.173470 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:32:12.173470 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 16:32:12.173470 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:32:12.173470 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 16:32:12.173470 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:32:12.173470 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:32:12.173470 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:32:12.173470 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:32:12.173470 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:32:12.173470 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:32:12.173470 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:32:12.173470 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:32:12.173470 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:32:12.173470 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:32:12.173470 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:32:12.173470 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:32:12.173470 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:32:12.174660 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:32:12.174660 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:32:12.174660 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 16:32:12.174660 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 16:32:12.174660 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 16:32:12.174660 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 16:32:12.174660 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 16:32:12.174660 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 16:32:12.174660 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 16:32:12.174660 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:32:12.174660 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 637/1253] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:32:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:32:12.179852 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbb83fb-390c-4381-8282-6f73674d3c51 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:12.241203 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:12.288866 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbb83fb-390c-4381-8282-6f73674d3c51 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:12.325327 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1abd408-b20a-4417-b2f6-7cc2dca59dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:12.326007 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1abd408-b20a-4417-b2f6-7cc2dca59dfa None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:12.326290 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1abd408-b20a-4417-b2f6-7cc2dca59dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:12.326569 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1abd408-b20a-4417-b2f6-7cc2dca59dfa None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:12.327448 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1abd408-b20a-4417-b2f6-7cc2dca59dfa None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:32:12.327756 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1abd408-b20a-4417-b2f6-7cc2dca59dfa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:12.329223 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1abd408-b20a-4417-b2f6-7cc2dca59dfa None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:12.348602 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 617/1254] 10.222.0.36 () {56 vars in 1143 bytes} [Thu May 19 16:32:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:32:12.354567 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15803e1d-4d35-46d8-8352-fa86a8b6bd36 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:12.464780 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15803e1d-4d35-46d8-8352-fa86a8b6bd36 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:12.500841 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb9d30-390a-425c-8a9e-aa4136d987ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:12.501541 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb9d30-390a-425c-8a9e-aa4136d987ca None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:12.501956 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb9d30-390a-425c-8a9e-aa4136d987ca None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:12.502118 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb9d30-390a-425c-8a9e-aa4136d987ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:12.561529 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:12.575074 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfb9d30-390a-425c-8a9e-aa4136d987ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:12.576540 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfb9d30-390a-425c-8a9e-aa4136d987ca None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:12.691701 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 638/1255] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:32:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 16:32:12.698232 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1abd408-b20a-4417-b2f6-7cc2dca59dfa None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:12.807316 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1abd408-b20a-4417-b2f6-7cc2dca59dfa None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:12.822591 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:12.843886 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c961f6b7-cbef-4868-91fa-92f600e0c381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:12.844673 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961f6b7-cbef-4868-91fa-92f600e0c381 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:12.844958 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961f6b7-cbef-4868-91fa-92f600e0c381 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:12.845272 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961f6b7-cbef-4868-91fa-92f600e0c381 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:12.855120 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c961f6b7-cbef-4868-91fa-92f600e0c381 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:12.856561 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c961f6b7-cbef-4868-91fa-92f600e0c381 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:12.874187 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-c961f6b7-cbef-4868-91fa-92f600e0c381 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:32:12.874187 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:32:12.874187 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:32:12.874187 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:32:12.874187 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:32:12.874187 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:32:12.874187 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:32:12.874187 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:32:12.874187 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:32:12.874187 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:32:12.874187 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:32:12.874187 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:32:12.874187 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:32:12.874187 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:32:12.874187 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:32:12.875283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:32:12.875283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:32:12.875283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:32:12.875283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:32:12.875283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:32:12.875283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:32:12.875283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:32:12.875283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:32:12.875283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:32:12.875283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:32:12.875283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:32:12.875283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:32:12.875283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:32:12.875283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:32:12.875283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:32:12.875283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:32:12.875283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:32:12.876582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:32:12.876582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:32:12.876582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:32:12.876582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:32:12.876582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 618/1256] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:32:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:32:12.881088 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb9d30-390a-425c-8a9e-aa4136d987ca None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:12.969800 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:12.987043 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb9d30-390a-425c-8a9e-aa4136d987ca None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:13.023946 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e12d8-144e-452d-a49c-89ea77eff3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:13.024609 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e12d8-144e-452d-a49c-89ea77eff3eb None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:13.024899 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e12d8-144e-452d-a49c-89ea77eff3eb None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:13.025201 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e12d8-144e-452d-a49c-89ea77eff3eb None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:13.025935 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770e12d8-144e-452d-a49c-89ea77eff3eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:32:13.026246 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770e12d8-144e-452d-a49c-89ea77eff3eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:13.027632 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770e12d8-144e-452d-a49c-89ea77eff3eb None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:13.039022 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 639/1257] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:32:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:32:13.044626 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c961f6b7-cbef-4868-91fa-92f600e0c381 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:13.142683 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c961f6b7-cbef-4868-91fa-92f600e0c381 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:13.176140 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a58fc8b-6026-4959-9939-bebfeecfd0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:13.176934 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58fc8b-6026-4959-9939-bebfeecfd0af None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:13.177251 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58fc8b-6026-4959-9939-bebfeecfd0af None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:13.177535 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58fc8b-6026-4959-9939-bebfeecfd0af None admin] PATH_INFO: `/v3/projects/service` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:13.184844 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a58fc8b-6026-4959-9939-bebfeecfd0af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:13.186339 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a58fc8b-6026-4959-9939-bebfeecfd0af None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:13.199998 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-6a58fc8b-6026-4959-9939-bebfeecfd0af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:32:13.199998 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:32:13.199998 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:32:13.199998 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:32:13.199998 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:32:13.199998 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:32:13.199998 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:32:13.199998 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:32:13.199998 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:32:13.199998 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:32:13.199998 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:32:13.199998 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:32:13.199998 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 16:32:13.199998 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 16:32:13.199998 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 16:32:13.201059 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 16:32:13.201059 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:32:13.201059 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:32:13.201059 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:32:13.201059 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:32:13.201059 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:32:13.201059 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:32:13.201059 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:32:13.201059 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:32:13.201059 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:32:13.201059 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:32:13.201059 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:32:13.201059 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:32:13.201059 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:32:13.201059 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:32:13.201059 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:32:13.201059 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:32:13.202191 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 16:32:13.202191 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 16:32:13.202191 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 16:32:13.202191 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 16:32:13.202191 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 16:32:13.202191 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 16:32:13.202191 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:32:13.202191 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:32:13.202191 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 619/1258] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:32:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:32:13.207084 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e12d8-144e-452d-a49c-89ea77eff3eb None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:13.309648 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e12d8-144e-452d-a49c-89ea77eff3eb None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:13.342715 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328f1529-8eb8-4cdd-8f8f-cbb8c9edeb19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:13.343372 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f1529-8eb8-4cdd-8f8f-cbb8c9edeb19 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:13.343661 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f1529-8eb8-4cdd-8f8f-cbb8c9edeb19 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:13.343938 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f1529-8eb8-4cdd-8f8f-cbb8c9edeb19 None admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:13.344728 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328f1529-8eb8-4cdd-8f8f-cbb8c9edeb19 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:32:13.345042 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328f1529-8eb8-4cdd-8f8f-cbb8c9edeb19 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:13.346530 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328f1529-8eb8-4cdd-8f8f-cbb8c9edeb19 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:13.359044 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 640/1259] 10.222.0.36 () {56 vars in 1148 bytes} [Thu May 19 16:32:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:32:13.364989 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a58fc8b-6026-4959-9939-bebfeecfd0af None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:13.392205 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:13.467417 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a58fc8b-6026-4959-9939-bebfeecfd0af None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:13.503861 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296300c0-0003-4485-9935-2f23cdfa5b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:13.504537 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296300c0-0003-4485-9935-2f23cdfa5b01 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:13.504826 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296300c0-0003-4485-9935-2f23cdfa5b01 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:13.505235 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296300c0-0003-4485-9935-2f23cdfa5b01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:13.506124 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296300c0-0003-4485-9935-2f23cdfa5b01 None admin] RBAC: Adding query filter params (role.id=e2915fe109244a07a9be96bd2b219fdd, scope.project.id=631c76794a124d6da22c626e590e4352, user.id=c6e158e5c87f4954aeb297f42dabdc16) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:32:13.506333 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296300c0-0003-4485-9935-2f23cdfa5b01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:13.507816 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296300c0-0003-4485-9935-2f23cdfa5b01 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:13.515089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 620/1260] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:32:13 2022] GET /identity/v3/role_assignments?user.id=c6e158e5c87f4954aeb297f42dabdc16&scope.project.id=631c76794a124d6da22c626e590e4352&role.id=e2915fe109244a07a9be96bd2b219fdd => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:32:15.805972 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87e15a-7e78-47e9-a020-d8e93aba6a49 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:15.809047 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87e15a-7e78-47e9-a020-d8e93aba6a49 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:15.809047 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87e15a-7e78-47e9-a020-d8e93aba6a49 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:15.809047 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 641/1261] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:32:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:32:15.813484 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d743d4a-24b8-4317-916c-24bc877054fa None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:15.813924 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d743d4a-24b8-4317-916c-24bc877054fa None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:15.814052 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d743d4a-24b8-4317-916c-24bc877054fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:15.831860 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-4d743d4a-24b8-4317-916c-24bc877054fa None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:32:15.833051 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-4d743d4a-24b8-4317-916c-24bc877054fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:32:15.899893 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 621/1262] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:32:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:32:15.922841 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec87e15a-7e78-47e9-a020-d8e93aba6a49 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:15.935501 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:16.034897 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec87e15a-7e78-47e9-a020-d8e93aba6a49 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:16.075213 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a2b6c9-94fd-4fd2-ab20-db2f80fd915f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:16.075965 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a2b6c9-94fd-4fd2-ab20-db2f80fd915f None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:16.076257 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a2b6c9-94fd-4fd2-ab20-db2f80fd915f None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:16.076540 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a2b6c9-94fd-4fd2-ab20-db2f80fd915f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:16.089783 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a2b6c9-94fd-4fd2-ab20-db2f80fd915f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:32:16.097881 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-46a2b6c9-94fd-4fd2-ab20-db2f80fd915f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:32:16.097881 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:32:16.097881 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:32:16.097881 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:32:16.097881 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:32:16.097881 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:32:16.097881 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:32:16.097881 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:32:16.097881 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:32:16.097881 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:32:16.097881 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:32:16.097881 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:32:16.097881 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:32:16.097881 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:32:16.097881 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:32:16.097881 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:32:16.099120 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:32:16.099120 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:32:16.099120 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:32:16.099120 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:32:16.099120 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:32:16.099120 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:32:16.099120 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:32:16.099120 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:32:16.099120 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:32:16.099120 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:32:16.099120 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:32:16.099120 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:32:16.099120 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:32:16.099120 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:32:16.099120 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:32:16.099120 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:32:16.100059 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:32:16.100059 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:32:16.100059 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:32:16.100059 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:32:16.100059 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:32:16.100059 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:32:16.100059 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:32:16.100059 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:32:16.100059 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:32:16.100059 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:32:16.100059 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:32:16.100059 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:32:16.100059 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 642/1263] 10.222.0.36 () {56 vars in 1096 bytes} [Thu May 19 16:32:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:32:16.105778 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d743d4a-24b8-4317-916c-24bc877054fa None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:16.127302 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:16.224838 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d743d4a-24b8-4317-916c-24bc877054fa None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:16.266209 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7796c-faf9-4add-aab1-bb2c63b9affd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:16.266923 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7796c-faf9-4add-aab1-bb2c63b9affd None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:16.267211 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7796c-faf9-4add-aab1-bb2c63b9affd None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:16.267495 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7796c-faf9-4add-aab1-bb2c63b9affd None admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:16.268315 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f7796c-faf9-4add-aab1-bb2c63b9affd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:32:16.268628 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f7796c-faf9-4add-aab1-bb2c63b9affd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:16.270159 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f7796c-faf9-4add-aab1-bb2c63b9affd None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:16.280634 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 622/1264] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:32:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:32:16.286934 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a2b6c9-94fd-4fd2-ab20-db2f80fd915f None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:16.389362 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:16.431870 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a2b6c9-94fd-4fd2-ab20-db2f80fd915f None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:16.469754 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c0b91-7574-4f44-a81f-94e749b41706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:16.470582 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c0b91-7574-4f44-a81f-94e749b41706 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:16.470865 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c0b91-7574-4f44-a81f-94e749b41706 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:16.471141 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c0b91-7574-4f44-a81f-94e749b41706 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:16.484202 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929c0b91-7574-4f44-a81f-94e749b41706 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:16.485761 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929c0b91-7574-4f44-a81f-94e749b41706 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:16.504942 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-929c0b91-7574-4f44-a81f-94e749b41706 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:32:16.504942 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:32:16.504942 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:32:16.504942 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:32:16.504942 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:32:16.504942 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:32:16.504942 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:32:16.504942 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:32:16.504942 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:32:16.504942 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:32:16.504942 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:32:16.504942 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:32:16.504942 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:32:16.504942 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:32:16.504942 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:32:16.506112 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:32:16.506112 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:32:16.506112 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:32:16.506112 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:32:16.506112 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:32:16.506112 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:32:16.506112 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:32:16.506112 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:32:16.506112 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:32:16.506112 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:32:16.506112 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:32:16.506112 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:32:16.506112 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:32:16.506112 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:32:16.506112 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:32:16.506112 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:32:16.506112 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:32:16.507231 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:32:16.507231 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:32:16.507231 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:32:16.507231 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:32:16.507231 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 643/1265] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:32:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:32:16.512517 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7796c-faf9-4add-aab1-bb2c63b9affd None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:16.605678 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:16.629922 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7796c-faf9-4add-aab1-bb2c63b9affd None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:16.673485 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6300d72a-545a-4f21-b4ea-aae59efc222c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:16.674313 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300d72a-545a-4f21-b4ea-aae59efc222c None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:16.674542 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300d72a-545a-4f21-b4ea-aae59efc222c None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:16.675006 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300d72a-545a-4f21-b4ea-aae59efc222c None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:16.675828 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6300d72a-545a-4f21-b4ea-aae59efc222c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:32:16.676186 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6300d72a-545a-4f21-b4ea-aae59efc222c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:16.679694 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6300d72a-545a-4f21-b4ea-aae59efc222c None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:16.697416 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 623/1266] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:32:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:32:16.707808 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c0b91-7574-4f44-a81f-94e749b41706 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:16.813361 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c0b91-7574-4f44-a81f-94e749b41706 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:16.832603 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:16.846785 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0126dc6f-eebe-4e8e-a6fa-8adec503be8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:16.847585 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0126dc6f-eebe-4e8e-a6fa-8adec503be8b None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:16.847872 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0126dc6f-eebe-4e8e-a6fa-8adec503be8b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:16.848152 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0126dc6f-eebe-4e8e-a6fa-8adec503be8b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:16.856032 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0126dc6f-eebe-4e8e-a6fa-8adec503be8b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:16.857513 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0126dc6f-eebe-4e8e-a6fa-8adec503be8b None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:16.871471 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-0126dc6f-eebe-4e8e-a6fa-8adec503be8b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 16:32:16.871471 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:32:16.871471 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:32:16.871471 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:32:16.871471 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:32:16.871471 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:32:16.871471 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:32:16.871471 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:32:16.871471 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:32:16.871471 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:32:16.871471 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:32:16.871471 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:32:16.871471 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 16:32:16.871471 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 16:32:16.871471 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 16:32:16.871471 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 16:32:16.872588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:32:16.872588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:32:16.872588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 16:32:16.872588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:32:16.872588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 16:32:16.872588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:32:16.872588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:32:16.872588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:32:16.872588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:32:16.872588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:32:16.872588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:32:16.872588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:32:16.872588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:32:16.872588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:32:16.872588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:32:16.872588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:32:16.872588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:32:16.872588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:32:16.878248 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:32:16.878248 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:32:16.878248 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 16:32:16.878248 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 16:32:16.878248 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 16:32:16.878248 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 16:32:16.878248 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 16:32:16.878248 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 16:32:16.878248 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 16:32:16.878248 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:32:16.878248 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 644/1267] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:32:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:32:16.878884 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6300d72a-545a-4f21-b4ea-aae59efc222c None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:16.980739 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6300d72a-545a-4f21-b4ea-aae59efc222c None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:17.019295 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef5fe9-aab2-4e7d-96ce-41f59f5e5f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:17.019968 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef5fe9-aab2-4e7d-96ce-41f59f5e5f5d None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:17.020257 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef5fe9-aab2-4e7d-96ce-41f59f5e5f5d None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:17.020542 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef5fe9-aab2-4e7d-96ce-41f59f5e5f5d None admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:17.021426 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaef5fe9-aab2-4e7d-96ce-41f59f5e5f5d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:32:17.021740 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaef5fe9-aab2-4e7d-96ce-41f59f5e5f5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:17.023226 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaef5fe9-aab2-4e7d-96ce-41f59f5e5f5d None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:17.032531 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:17.041554 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 624/1268] 10.222.0.36 () {56 vars in 1143 bytes} [Thu May 19 16:32:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:32:17.047179 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0126dc6f-eebe-4e8e-a6fa-8adec503be8b None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:17.162447 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0126dc6f-eebe-4e8e-a6fa-8adec503be8b None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:17.197078 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe274967-69bf-4321-912a-76cffe142875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:17.197913 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe274967-69bf-4321-912a-76cffe142875 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:17.198197 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe274967-69bf-4321-912a-76cffe142875 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:17.198475 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe274967-69bf-4321-912a-76cffe142875 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:17.210298 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe274967-69bf-4321-912a-76cffe142875 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:17.212480 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe274967-69bf-4321-912a-76cffe142875 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:17.229600 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-fe274967-69bf-4321-912a-76cffe142875 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:32:17.229600 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:32:17.229600 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:32:17.229600 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:32:17.229600 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:32:17.229600 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:32:17.229600 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:32:17.229600 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:32:17.229600 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:32:17.229600 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:32:17.229600 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:32:17.229600 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:32:17.229600 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:32:17.229600 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:32:17.229600 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:32:17.230660 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:32:17.230660 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:32:17.230660 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:32:17.230660 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:32:17.230660 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:32:17.230660 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:32:17.230660 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:32:17.230660 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:32:17.230660 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:32:17.230660 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:32:17.230660 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:32:17.230660 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:32:17.230660 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:32:17.230660 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:32:17.230660 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:32:17.230660 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:32:17.230660 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:32:17.232044 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:32:17.232044 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:32:17.232044 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:32:17.232044 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:32:17.232044 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 645/1269] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:32:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:32:17.236551 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef5fe9-aab2-4e7d-96ce-41f59f5e5f5d None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:17.337916 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef5fe9-aab2-4e7d-96ce-41f59f5e5f5d None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:17.371224 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af554ed-ac78-4e2f-a904-35a3f2d74c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:17.371895 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af554ed-ac78-4e2f-a904-35a3f2d74c7d None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:17.372184 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af554ed-ac78-4e2f-a904-35a3f2d74c7d None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:17.372465 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af554ed-ac78-4e2f-a904-35a3f2d74c7d None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:17.373259 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af554ed-ac78-4e2f-a904-35a3f2d74c7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:32:17.373574 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af554ed-ac78-4e2f-a904-35a3f2d74c7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:17.374958 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af554ed-ac78-4e2f-a904-35a3f2d74c7d None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:17.386857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 625/1270] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:32:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:32:17.392723 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe274967-69bf-4321-912a-76cffe142875 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:17.424242 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:17.491629 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe274967-69bf-4321-912a-76cffe142875 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:17.525527 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0792b0ef-485b-4603-afe2-aa415da0d607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:17.526319 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792b0ef-485b-4603-afe2-aa415da0d607 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:17.526610 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792b0ef-485b-4603-afe2-aa415da0d607 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:17.526889 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792b0ef-485b-4603-afe2-aa415da0d607 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:17.533392 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0792b0ef-485b-4603-afe2-aa415da0d607 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:17.534851 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0792b0ef-485b-4603-afe2-aa415da0d607 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:17.548671 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-0792b0ef-485b-4603-afe2-aa415da0d607 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:32:17.548671 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:32:17.548671 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:32:17.548671 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:32:17.548671 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:32:17.548671 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:32:17.548671 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:32:17.548671 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:32:17.548671 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:32:17.548671 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:32:17.548671 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:32:17.548671 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:32:17.548671 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 16:32:17.548671 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 16:32:17.548671 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 16:32:17.549766 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 16:32:17.549766 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:32:17.549766 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:32:17.549766 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:32:17.549766 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:32:17.549766 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:32:17.549766 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:32:17.549766 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:32:17.549766 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:32:17.549766 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:32:17.549766 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:32:17.549766 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:32:17.549766 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:32:17.549766 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:32:17.549766 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:32:17.549766 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:32:17.549766 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:32:17.550867 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 16:32:17.550867 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 16:32:17.550867 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 16:32:17.550867 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 16:32:17.550867 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 16:32:17.550867 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 16:32:17.550867 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:32:17.550867 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:32:17.550867 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 646/1271] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:32:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:32:17.556209 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af554ed-ac78-4e2f-a904-35a3f2d74c7d None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:17.615279 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:17.663948 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af554ed-ac78-4e2f-a904-35a3f2d74c7d None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:17.698402 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3956e46e-d548-40b5-b192-c1e97d588675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:17.699074 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956e46e-d548-40b5-b192-c1e97d588675 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:17.699362 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956e46e-d548-40b5-b192-c1e97d588675 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:17.699641 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956e46e-d548-40b5-b192-c1e97d588675 None admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:17.700436 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3956e46e-d548-40b5-b192-c1e97d588675 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:32:17.700750 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3956e46e-d548-40b5-b192-c1e97d588675 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:17.702225 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3956e46e-d548-40b5-b192-c1e97d588675 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:17.714500 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 626/1272] 10.222.0.36 () {56 vars in 1148 bytes} [Thu May 19 16:32:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:32:17.720860 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0792b0ef-485b-4603-afe2-aa415da0d607 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:17.813111 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:17.822655 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0792b0ef-485b-4603-afe2-aa415da0d607 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:17.857134 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39c368-8294-4d6e-9539-daed77873ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:17.858060 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39c368-8294-4d6e-9539-daed77873ce4 None admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:17.858345 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39c368-8294-4d6e-9539-daed77873ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:17.858624 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39c368-8294-4d6e-9539-daed77873ce4 None admin] PATH_INFO: `/v3/projects/631c76794a124d6da22c626e590e4352/users/c6e158e5c87f4954aeb297f42dabdc16/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:17.860591 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39c368-8294-4d6e-9539-daed77873ce4 None admin] RBAC: Authorizing `identity:create_grant(project_id=631c76794a124d6da22c626e590e4352, user_id=c6e158e5c87f4954aeb297f42dabdc16, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:17.862080 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39c368-8294-4d6e-9539-daed77873ce4 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:17.875458 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 647/1273] 10.222.0.36 () {58 vars in 1439 bytes} [Thu May 19 16:32:17 2022] PUT /identity/v3/projects/631c76794a124d6da22c626e590e4352/users/c6e158e5c87f4954aeb297f42dabdc16/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:32:20.109845 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7694fa6-4c5e-453f-bf7b-a823ce4c0a1a None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:20.110662 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7694fa6-4c5e-453f-bf7b-a823ce4c0a1a None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:20.110817 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7694fa6-4c5e-453f-bf7b-a823ce4c0a1a None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:20.112236 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 627/1274] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:32:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 16:32:20.117791 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767e4d4-e7b8-4efe-87ab-ed31eb49d0d7 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:20.118276 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767e4d4-e7b8-4efe-87ab-ed31eb49d0d7 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:20.118415 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767e4d4-e7b8-4efe-87ab-ed31eb49d0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:20.135974 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-8767e4d4-e7b8-4efe-87ab-ed31eb49d0d7 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:32:20.137299 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-8767e4d4-e7b8-4efe-87ab-ed31eb49d0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:32:20.206858 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 648/1275] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:32:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:32:20.230700 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7694fa6-4c5e-453f-bf7b-a823ce4c0a1a None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:20.339637 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7694fa6-4c5e-453f-bf7b-a823ce4c0a1a None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:20.358611 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:20.375999 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433dcebb-0c0a-4904-a93a-c1e301ed1363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:20.376799 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433dcebb-0c0a-4904-a93a-c1e301ed1363 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:20.377081 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433dcebb-0c0a-4904-a93a-c1e301ed1363 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:20.377386 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433dcebb-0c0a-4904-a93a-c1e301ed1363 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:20.388489 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433dcebb-0c0a-4904-a93a-c1e301ed1363 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 16:32:20.396404 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-433dcebb-0c0a-4904-a93a-c1e301ed1363 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:32:20.396404 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:32:20.396404 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:32:20.396404 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:32:20.396404 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:32:20.396404 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:32:20.396404 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:32:20.396404 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:32:20.396404 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:32:20.396404 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:32:20.396404 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:32:20.396404 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:32:20.396404 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 16:32:20.396404 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 16:32:20.396404 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 16:32:20.396404 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 16:32:20.397569 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 16:32:20.397569 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 16:32:20.397569 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 16:32:20.397569 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 16:32:20.397569 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:32:20.397569 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:32:20.397569 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:32:20.397569 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:32:20.397569 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:32:20.397569 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:32:20.397569 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:32:20.397569 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:32:20.397569 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:32:20.397569 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:32:20.397569 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:32:20.397569 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:32:20.399069 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:32:20.399069 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:32:20.399069 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:32:20.399069 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:32:20.399069 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 16:32:20.399069 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 16:32:20.399069 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 16:32:20.399069 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 16:32:20.399069 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 16:32:20.399069 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 16:32:20.399069 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 16:32:20.399069 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:32:20.399069 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 628/1276] 10.222.0.36 () {56 vars in 1096 bytes} [Thu May 19 16:32:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:32:20.404506 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8767e4d4-e7b8-4efe-87ab-ed31eb49d0d7 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:20.488275 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:20.508761 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8767e4d4-e7b8-4efe-87ab-ed31eb49d0d7 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:20.543779 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ea1045-98e1-49eb-905e-63cc6d1ea07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:20.544474 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea1045-98e1-49eb-905e-63cc6d1ea07e None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:20.544759 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea1045-98e1-49eb-905e-63cc6d1ea07e None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:20.545036 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea1045-98e1-49eb-905e-63cc6d1ea07e None admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:20.545835 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ea1045-98e1-49eb-905e-63cc6d1ea07e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:32:20.546144 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ea1045-98e1-49eb-905e-63cc6d1ea07e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:20.547538 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ea1045-98e1-49eb-905e-63cc6d1ea07e None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:20.557657 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 649/1277] 10.222.0.36 () {56 vars in 1099 bytes} [Thu May 19 16:32:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:32:20.563542 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433dcebb-0c0a-4904-a93a-c1e301ed1363 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:20.670652 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433dcebb-0c0a-4904-a93a-c1e301ed1363 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:20.705690 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94622dd-51f5-40bc-95fa-0f0095fd0188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:20.706485 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94622dd-51f5-40bc-95fa-0f0095fd0188 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:20.706768 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94622dd-51f5-40bc-95fa-0f0095fd0188 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:20.707064 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94622dd-51f5-40bc-95fa-0f0095fd0188 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:20.718813 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94622dd-51f5-40bc-95fa-0f0095fd0188 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:20.720261 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94622dd-51f5-40bc-95fa-0f0095fd0188 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:20.737266 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-d94622dd-51f5-40bc-95fa-0f0095fd0188 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:32:20.737266 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:32:20.737266 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:32:20.737266 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:32:20.737266 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:32:20.737266 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:32:20.737266 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:32:20.737266 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:32:20.737266 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:32:20.737266 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:32:20.737266 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:32:20.737266 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:32:20.737266 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:32:20.737266 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:32:20.737266 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:32:20.738955 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:32:20.738955 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:32:20.738955 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:32:20.738955 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:32:20.738955 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:32:20.738955 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:32:20.738955 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:32:20.738955 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:32:20.738955 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:32:20.738955 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:32:20.738955 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:32:20.738955 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:32:20.738955 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:32:20.738955 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:32:20.738955 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:32:20.738955 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:32:20.738955 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:32:20.740415 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:32:20.740415 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:32:20.740415 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:32:20.740415 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:32:20.740415 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 629/1278] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:32:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 16:32:20.744292 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ea1045-98e1-49eb-905e-63cc6d1ea07e None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:20.842092 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ea1045-98e1-49eb-905e-63cc6d1ea07e None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:20.881836 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe013770-33f1-4c6d-a136-94549430f5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:20.882499 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe013770-33f1-4c6d-a136-94549430f5e3 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:20.883315 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe013770-33f1-4c6d-a136-94549430f5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:20.883596 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe013770-33f1-4c6d-a136-94549430f5e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:20.884322 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe013770-33f1-4c6d-a136-94549430f5e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:32:20.884634 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe013770-33f1-4c6d-a136-94549430f5e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:20.886048 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe013770-33f1-4c6d-a136-94549430f5e3 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:20.899704 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 650/1279] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:32:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:32:20.905309 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94622dd-51f5-40bc-95fa-0f0095fd0188 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:20.938547 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:21.007605 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94622dd-51f5-40bc-95fa-0f0095fd0188 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:21.042438 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d442f32d-806c-40b2-8ce8-4f5e6b25ae1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:21.043325 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442f32d-806c-40b2-8ce8-4f5e6b25ae1d None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:21.043670 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442f32d-806c-40b2-8ce8-4f5e6b25ae1d None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:21.043974 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442f32d-806c-40b2-8ce8-4f5e6b25ae1d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:21.052062 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d442f32d-806c-40b2-8ce8-4f5e6b25ae1d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:21.053559 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d442f32d-806c-40b2-8ce8-4f5e6b25ae1d None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:21.067877 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-d442f32d-806c-40b2-8ce8-4f5e6b25ae1d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 16:32:21.067877 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:32:21.067877 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:32:21.067877 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:32:21.067877 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:32:21.067877 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:32:21.067877 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:32:21.067877 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:32:21.067877 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:32:21.067877 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:32:21.067877 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:32:21.067877 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:32:21.067877 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 16:32:21.067877 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 16:32:21.067877 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 16:32:21.067877 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 16:32:21.068994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:32:21.068994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:32:21.068994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 16:32:21.068994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:32:21.068994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 16:32:21.068994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 16:32:21.068994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:32:21.068994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:32:21.068994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:32:21.068994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:32:21.068994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:32:21.068994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:32:21.068994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:32:21.068994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:32:21.068994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:32:21.068994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:32:21.068994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:32:21.068994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:32:21.070236 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:32:21.070236 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:32:21.070236 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 16:32:21.070236 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 16:32:21.070236 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 16:32:21.070236 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 16:32:21.070236 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 16:32:21.070236 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 16:32:21.070236 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 16:32:21.070236 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:32:21.070236 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 630/1280] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:32:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:32:21.075079 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe013770-33f1-4c6d-a136-94549430f5e3 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:21.080599 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:21.175769 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe013770-33f1-4c6d-a136-94549430f5e3 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:21.210673 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd3676-5d11-454f-a0cf-c9cfeef409ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:21.211387 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd3676-5d11-454f-a0cf-c9cfeef409ca None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:21.211669 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd3676-5d11-454f-a0cf-c9cfeef409ca None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:21.211942 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd3676-5d11-454f-a0cf-c9cfeef409ca None admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:21.212793 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defd3676-5d11-454f-a0cf-c9cfeef409ca None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:32:21.213099 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defd3676-5d11-454f-a0cf-c9cfeef409ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:21.214586 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defd3676-5d11-454f-a0cf-c9cfeef409ca None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:21.233465 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 651/1281] 10.222.0.36 () {56 vars in 1143 bytes} [Thu May 19 16:32:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:32:21.239572 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d442f32d-806c-40b2-8ce8-4f5e6b25ae1d None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:21.335406 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:21.347562 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d442f32d-806c-40b2-8ce8-4f5e6b25ae1d None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:21.391927 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7c78b3-debd-493a-9032-f658cef864cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:21.392606 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c78b3-debd-493a-9032-f658cef864cc None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:21.392889 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c78b3-debd-493a-9032-f658cef864cc None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:21.393198 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c78b3-debd-493a-9032-f658cef864cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:21.479639 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7c78b3-debd-493a-9032-f658cef864cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:21.481135 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7c78b3-debd-493a-9032-f658cef864cc None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:21.575604 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:21.587144 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 631/1282] 10.222.0.36 () {58 vars in 1282 bytes} [Thu May 19 16:32:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 16:32:21.593214 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd3676-5d11-454f-a0cf-c9cfeef409ca None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:21.659615 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:21.700559 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd3676-5d11-454f-a0cf-c9cfeef409ca None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:21.738438 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b3120-e2d4-4824-94b3-49d89b2c2b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:21.739247 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b3120-e2d4-4824-94b3-49d89b2c2b1a None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:21.739533 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b3120-e2d4-4824-94b3-49d89b2c2b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:21.739812 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b3120-e2d4-4824-94b3-49d89b2c2b1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:21.750824 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27b3120-e2d4-4824-94b3-49d89b2c2b1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:21.752298 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27b3120-e2d4-4824-94b3-49d89b2c2b1a None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:21.770890 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-d27b3120-e2d4-4824-94b3-49d89b2c2b1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:32:21.770890 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:32:21.770890 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:32:21.770890 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:32:21.770890 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:32:21.770890 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:32:21.770890 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:32:21.770890 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:32:21.770890 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:32:21.770890 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:32:21.770890 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:32:21.770890 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:32:21.770890 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 16:32:21.770890 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 16:32:21.770890 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 16:32:21.772524 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 16:32:21.772524 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:32:21.772524 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:32:21.772524 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:32:21.772524 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:32:21.772524 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:32:21.772524 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:32:21.772524 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:32:21.772524 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:32:21.772524 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:32:21.772524 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:32:21.772524 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:32:21.772524 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:32:21.772524 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:32:21.772524 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:32:21.772524 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:32:21.772524 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:32:21.774425 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 16:32:21.774425 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 16:32:21.774425 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 16:32:21.774425 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:32:21.774425 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 652/1283] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:32:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:32:21.779454 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7c78b3-debd-493a-9032-f658cef864cc None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:21.880264 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7c78b3-debd-493a-9032-f658cef864cc None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:21.916099 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd2dfe-aa1a-4372-9bcb-4af7092ed101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:21.916771 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd2dfe-aa1a-4372-9bcb-4af7092ed101 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:21.917057 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd2dfe-aa1a-4372-9bcb-4af7092ed101 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:21.917363 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd2dfe-aa1a-4372-9bcb-4af7092ed101 None admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:21.918093 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcd2dfe-aa1a-4372-9bcb-4af7092ed101 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:32:21.918403 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcd2dfe-aa1a-4372-9bcb-4af7092ed101 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:21.919777 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcd2dfe-aa1a-4372-9bcb-4af7092ed101 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:21.932123 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 632/1284] 10.222.0.36 () {56 vars in 1109 bytes} [Thu May 19 16:32:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:32:21.937930 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b3120-e2d4-4824-94b3-49d89b2c2b1a None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:22.037545 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b3120-e2d4-4824-94b3-49d89b2c2b1a None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:22.053230 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:22.077336 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576f6cf-f311-4927-aefe-e8b7e0e407e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:22.078148 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576f6cf-f311-4927-aefe-e8b7e0e407e4 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:22.078436 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576f6cf-f311-4927-aefe-e8b7e0e407e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:22.078719 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576f6cf-f311-4927-aefe-e8b7e0e407e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:22.085666 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2576f6cf-f311-4927-aefe-e8b7e0e407e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:22.087173 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2576f6cf-f311-4927-aefe-e8b7e0e407e4 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:22.101517 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application [None req-2576f6cf-f311-4927-aefe-e8b7e0e407e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:32:22.101517 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:32:22.101517 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:32:22.101517 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:32:22.101517 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:32:22.101517 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:32:22.101517 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:32:22.101517 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:32:22.101517 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:32:22.101517 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:32:22.101517 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:32:22.101517 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:32:22.101517 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 16:32:22.101517 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 16:32:22.101517 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 16:32:22.102713 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 16:32:22.102713 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:32:22.102713 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:32:22.102713 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:32:22.102713 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:32:22.102713 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:32:22.102713 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:32:22.102713 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:32:22.102713 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application with Lock( May 19 16:32:22.102713 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:32:22.102713 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._enter() May 19 16:32:22.102713 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:32:22.102713 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:32:22.102713 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:32:22.102713 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.creator() May 19 16:32:22.102713 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:32:22.102713 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application created_value = creator( May 19 16:32:22.103712 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 16:32:22.103712 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 16:32:22.103712 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 16:32:22.103712 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 16:32:22.103712 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 16:32:22.103712 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 16:32:22.103712 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 16:32:22.103712 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: ERROR keystone.server.flask.application May 19 16:32:22.103712 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 653/1285] 10.222.0.36 () {56 vars in 1111 bytes} [Thu May 19 16:32:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:32:22.109351 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd2dfe-aa1a-4372-9bcb-4af7092ed101 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:22.160612 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:22.213845 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd2dfe-aa1a-4372-9bcb-4af7092ed101 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:22.247986 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b66ea01-a922-4f81-9828-79086cd5019e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:22.248655 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b66ea01-a922-4f81-9828-79086cd5019e None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:22.248943 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b66ea01-a922-4f81-9828-79086cd5019e None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:22.249844 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b66ea01-a922-4f81-9828-79086cd5019e None admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:22.250049 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b66ea01-a922-4f81-9828-79086cd5019e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:32:22.250363 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b66ea01-a922-4f81-9828-79086cd5019e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:22.251825 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b66ea01-a922-4f81-9828-79086cd5019e None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:22.265373 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 633/1286] 10.222.0.36 () {56 vars in 1148 bytes} [Thu May 19 16:32:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:32:22.271748 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576f6cf-f311-4927-aefe-e8b7e0e407e4 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:22.377587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576f6cf-f311-4927-aefe-e8b7e0e407e4 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:22.412324 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20876175-6b53-4d04-84f3-0cac70955885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:22.412914 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20876175-6b53-4d04-84f3-0cac70955885 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:22.413215 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20876175-6b53-4d04-84f3-0cac70955885 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:22.413496 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20876175-6b53-4d04-84f3-0cac70955885 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:22.415062 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20876175-6b53-4d04-84f3-0cac70955885 None admin] RBAC: Adding query filter params (role.id=e2915fe109244a07a9be96bd2b219fdd, scope.project.id=631c76794a124d6da22c626e590e4352, user.id=c6e158e5c87f4954aeb297f42dabdc16) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:32:22.415374 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20876175-6b53-4d04-84f3-0cac70955885 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:22.416831 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20876175-6b53-4d04-84f3-0cac70955885 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:22.423146 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 654/1287] 10.222.0.36 () {56 vars in 1374 bytes} [Thu May 19 16:32:22 2022] GET /identity/v3/role_assignments?user.id=c6e158e5c87f4954aeb297f42dabdc16&scope.project.id=631c76794a124d6da22c626e590e4352&role.id=e2915fe109244a07a9be96bd2b219fdd => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:32:24.700359 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e213d58a-7551-4e6f-bce0-73726a8a16f1 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:24.700902 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e213d58a-7551-4e6f-bce0-73726a8a16f1 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:24.700902 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e213d58a-7551-4e6f-bce0-73726a8a16f1 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:24.702674 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 634/1288] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:32:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:32:24.710780 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a675f-0fa0-42b7-959d-4c8e6ed4bd66 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:24.712134 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a675f-0fa0-42b7-959d-4c8e6ed4bd66 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:24.712539 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a675f-0fa0-42b7-959d-4c8e6ed4bd66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:24.737778 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-c38a675f-0fa0-42b7-959d-4c8e6ed4bd66 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:32:24.739010 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-c38a675f-0fa0-42b7-959d-4c8e6ed4bd66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:32:24.769889 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:24.807940 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 655/1289] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:32:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 16:32:24.830808 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e213d58a-7551-4e6f-bce0-73726a8a16f1 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:24.934673 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e213d58a-7551-4e6f-bce0-73726a8a16f1 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:24.945765 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:24.969840 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d64321-a2a3-4681-b613-acb0365c7fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:24.970561 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d64321-a2a3-4681-b613-acb0365c7fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:24.970845 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d64321-a2a3-4681-b613-acb0365c7fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:24.971125 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d64321-a2a3-4681-b613-acb0365c7fc8 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:24.971785 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d64321-a2a3-4681-b613-acb0365c7fc8 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:24.973200 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d64321-a2a3-4681-b613-acb0365c7fc8 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:24.984743 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-14d64321-a2a3-4681-b613-acb0365c7fc8 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 16:32:24.984743 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:32:24.984743 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 16:32:24.984743 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 16:32:24.984743 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 16:32:24.984743 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 16:32:24.984743 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 16:32:24.984743 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:32:24.984743 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 16:32:24.984743 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:32:24.984743 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 16:32:24.984743 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:32:24.984743 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:32:24.984743 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:32:24.984743 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:32:24.984743 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:32:24.984743 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:32:24.986031 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:32:24.986031 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:32:24.986031 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:32:24.986031 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:32:24.986031 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:32:24.986031 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:32:24.986031 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 16:32:24.986031 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 16:32:24.986031 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 16:32:24.986031 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 16:32:24.986031 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:32:24.986031 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:32:24.986031 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:32:24.986031 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:32:24.986031 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:32:24.986031 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:32:24.987100 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:32:24.987100 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:32:24.987100 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:32:24.987100 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:32:24.987100 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:32:24.987100 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:32:24.987100 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:32:24.987100 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:32:24.987100 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:32:24.987100 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:32:24.987100 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 16:32:24.987100 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:32:24.987100 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 16:32:24.987100 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:32:24.989306 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 635/1290] 10.222.0.36 () {56 vars in 1117 bytes} [Thu May 19 16:32:24 2022] GET /identity/v3/services/placement => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:32:24.992732 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a675f-0fa0-42b7-959d-4c8e6ed4bd66 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:25.091138 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a675f-0fa0-42b7-959d-4c8e6ed4bd66 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:25.124454 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e08be-e9f8-47f4-b3bd-a2ef5bc3646f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:25.125128 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e08be-e9f8-47f4-b3bd-a2ef5bc3646f None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:25.125526 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e08be-e9f8-47f4-b3bd-a2ef5bc3646f None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:25.125807 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e08be-e9f8-47f4-b3bd-a2ef5bc3646f None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:25.126526 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2e08be-e9f8-47f4-b3bd-a2ef5bc3646f None admin] RBAC: Adding query filter params (name=placement) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:32:25.126841 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2e08be-e9f8-47f4-b3bd-a2ef5bc3646f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:25.128257 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2e08be-e9f8-47f4-b3bd-a2ef5bc3646f None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:25.133782 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 656/1291] 10.222.0.36 () {56 vars in 1116 bytes} [Thu May 19 16:32:24 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:32:25.139911 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d64321-a2a3-4681-b613-acb0365c7fc8 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:25.240577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d64321-a2a3-4681-b613-acb0365c7fc8 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:25.274054 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec823759-7c6f-4233-9466-f05ba6d3c3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:25.274711 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec823759-7c6f-4233-9466-f05ba6d3c3fa None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:25.274992 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec823759-7c6f-4233-9466-f05ba6d3c3fa None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:25.275264 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec823759-7c6f-4233-9466-f05ba6d3c3fa None admin] PATH_INFO: `/v3/services` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:25.275970 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec823759-7c6f-4233-9466-f05ba6d3c3fa None admin] RBAC: Adding query filter params (type=placement) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:32:25.276276 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec823759-7c6f-4233-9466-f05ba6d3c3fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:25.277673 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec823759-7c6f-4233-9466-f05ba6d3c3fa None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:25.282077 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 636/1292] 10.222.0.36 () {56 vars in 1116 bytes} [Thu May 19 16:32:25 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:32:27.516503 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1648119d-1699-48b5-a2d2-2157e7e12102 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:27.517113 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1648119d-1699-48b5-a2d2-2157e7e12102 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:27.517113 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1648119d-1699-48b5-a2d2-2157e7e12102 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:27.518474 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 657/1293] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:32:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:32:27.524529 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd268a-386e-4c53-9bcd-1a9c43befcb0 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:27.525006 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd268a-386e-4c53-9bcd-1a9c43befcb0 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:27.525113 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd268a-386e-4c53-9bcd-1a9c43befcb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:27.542812 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-5fbd268a-386e-4c53-9bcd-1a9c43befcb0 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:32:27.544022 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-5fbd268a-386e-4c53-9bcd-1a9c43befcb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:32:27.572861 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:27.614971 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 637/1294] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:32:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:32:27.641107 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1648119d-1699-48b5-a2d2-2157e7e12102 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:27.676617 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:27.749341 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1648119d-1699-48b5-a2d2-2157e7e12102 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:27.783382 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04243a1f-8fc4-43e2-97da-7b5eaf1a1fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:27.783971 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04243a1f-8fc4-43e2-97da-7b5eaf1a1fcb None admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:27.784262 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04243a1f-8fc4-43e2-97da-7b5eaf1a1fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:27.784547 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04243a1f-8fc4-43e2-97da-7b5eaf1a1fcb None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:27.785317 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04243a1f-8fc4-43e2-97da-7b5eaf1a1fcb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:27.786717 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04243a1f-8fc4-43e2-97da-7b5eaf1a1fcb None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:27.799703 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 658/1295] 10.222.0.36 () {60 vars in 1141 bytes} [Thu May 19 16:32:27 2022] POST /identity/v3/services => generated 246 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:32:30.045696 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a488037-9aec-46f6-a546-f6012722691d None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:30.046214 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a488037-9aec-46f6-a546-f6012722691d None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:30.046325 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a488037-9aec-46f6-a546-f6012722691d None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:30.047646 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 638/1296] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:32:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:32:30.053096 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae2dc9-28e0-42df-81df-d0192cfde9c5 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:30.053561 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae2dc9-28e0-42df-81df-d0192cfde9c5 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:30.053691 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae2dc9-28e0-42df-81df-d0192cfde9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:30.070788 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-6cae2dc9-28e0-42df-81df-d0192cfde9c5 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:32:30.072001 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-6cae2dc9-28e0-42df-81df-d0192cfde9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:32:30.155289 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 659/1297] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:32:30 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 16:32:30.177972 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a488037-9aec-46f6-a546-f6012722691d None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:30.278487 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a488037-9aec-46f6-a546-f6012722691d None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:30.311589 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f0e0a-601e-49de-9afe-792cb2e9d47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:30.312387 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f0e0a-601e-49de-9afe-792cb2e9d47d None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:30.312672 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f0e0a-601e-49de-9afe-792cb2e9d47d None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:30.312954 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f0e0a-601e-49de-9afe-792cb2e9d47d None admin] PATH_INFO: `/v3/services/placement` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:30.313651 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6f0e0a-601e-49de-9afe-792cb2e9d47d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:30.315028 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6f0e0a-601e-49de-9afe-792cb2e9d47d None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:30.327189 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-5f6f0e0a-601e-49de-9afe-792cb2e9d47d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 16:32:30.327189 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:32:30.327189 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 16:32:30.327189 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 16:32:30.327189 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 16:32:30.327189 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 16:32:30.327189 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 16:32:30.327189 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:32:30.327189 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 16:32:30.327189 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:32:30.327189 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 16:32:30.327189 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:32:30.327189 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:32:30.327189 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:32:30.327189 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:32:30.327189 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:32:30.327189 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:32:30.328449 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:32:30.328449 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:32:30.328449 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:32:30.328449 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:32:30.328449 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:32:30.328449 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:32:30.328449 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 16:32:30.328449 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 16:32:30.328449 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 16:32:30.328449 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 16:32:30.328449 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:32:30.328449 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:32:30.328449 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:32:30.328449 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:32:30.328449 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:32:30.328449 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:32:30.329586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:32:30.329586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:32:30.329586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:32:30.329586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:32:30.329586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:32:30.329586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:32:30.329586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:32:30.329586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:32:30.329586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:32:30.329586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:32:30.329586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 16:32:30.329586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:32:30.329586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 16:32:30.329586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:32:30.329586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 639/1298] 10.222.0.36 () {56 vars in 1117 bytes} [Thu May 19 16:32:30 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:32:30.334757 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cae2dc9-28e0-42df-81df-d0192cfde9c5 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:30.353769 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:30.452983 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cae2dc9-28e0-42df-81df-d0192cfde9c5 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:30.485720 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3406ea3b-81bf-42dc-aae4-5f9d92fec115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:30.486387 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406ea3b-81bf-42dc-aae4-5f9d92fec115 None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:30.486671 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406ea3b-81bf-42dc-aae4-5f9d92fec115 None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:30.486947 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406ea3b-81bf-42dc-aae4-5f9d92fec115 None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:30.487658 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3406ea3b-81bf-42dc-aae4-5f9d92fec115 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:32:30.487963 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3406ea3b-81bf-42dc-aae4-5f9d92fec115 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:30.489357 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3406ea3b-81bf-42dc-aae4-5f9d92fec115 None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:30.494952 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 660/1299] 10.222.0.36 () {56 vars in 1116 bytes} [Thu May 19 16:32:30 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:32:30.500966 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f0e0a-601e-49de-9afe-792cb2e9d47d None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:30.533479 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:30.604850 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f0e0a-601e-49de-9afe-792cb2e9d47d None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:30.642183 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2095df1-81b2-4536-a215-51c376a72f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:30.642898 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2095df1-81b2-4536-a215-51c376a72f0f None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:30.643176 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2095df1-81b2-4536-a215-51c376a72f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:30.643459 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2095df1-81b2-4536-a215-51c376a72f0f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:30.644266 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2095df1-81b2-4536-a215-51c376a72f0f None admin] RBAC: Adding query filter params (interface=public, service_id=afa88522add44aeea2e5a7cf8fd5f905, region_id=RegionOne) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:32:30.644582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2095df1-81b2-4536-a215-51c376a72f0f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:30.646000 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2095df1-81b2-4536-a215-51c376a72f0f None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:30.652946 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 640/1300] 10.222.0.36 () {56 vars in 1251 bytes} [Thu May 19 16:32:30 2022] GET /identity/v3/endpoints?service_id=afa88522add44aeea2e5a7cf8fd5f905&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:32:30.659553 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3406ea3b-81bf-42dc-aae4-5f9d92fec115 None admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:30.740954 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:30.760055 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3406ea3b-81bf-42dc-aae4-5f9d92fec115 None admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:30.793358 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c727a-8adc-4e49-aa08-8a506b08519b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:30.794015 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c727a-8adc-4e49-aa08-8a506b08519b None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:30.794299 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c727a-8adc-4e49-aa08-8a506b08519b None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:30.794573 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c727a-8adc-4e49-aa08-8a506b08519b None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:30.795254 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246c727a-8adc-4e49-aa08-8a506b08519b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:30.796625 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246c727a-8adc-4e49-aa08-8a506b08519b None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:30.803408 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 661/1301] 10.222.0.36 () {56 vars in 1087 bytes} [Thu May 19 16:32:30 2022] GET /identity/v3/services => generated 2191 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:32:33.059624 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b78bdf-5805-4a25-b9e8-77d185160e4d None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:33.060180 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b78bdf-5805-4a25-b9e8-77d185160e4d None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:33.060180 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b78bdf-5805-4a25-b9e8-77d185160e4d None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:33.061595 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 641/1302] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:32:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:32:33.066721 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08fdcfc-a0e6-4991-a0b2-bab7ff3d1df3 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:33.067142 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08fdcfc-a0e6-4991-a0b2-bab7ff3d1df3 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:33.067282 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08fdcfc-a0e6-4991-a0b2-bab7ff3d1df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:33.084256 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-f08fdcfc-a0e6-4991-a0b2-bab7ff3d1df3 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:32:33.086101 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-f08fdcfc-a0e6-4991-a0b2-bab7ff3d1df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:32:33.151245 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 662/1303] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:32:33 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 16:32:33.173779 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b78bdf-5805-4a25-b9e8-77d185160e4d None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:33.268332 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:33.275602 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b78bdf-5805-4a25-b9e8-77d185160e4d None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:33.308835 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1c1878-9919-4e0b-922e-f050ff341750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:33.309657 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c1878-9919-4e0b-922e-f050ff341750 None admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:33.309943 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c1878-9919-4e0b-922e-f050ff341750 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:33.310221 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c1878-9919-4e0b-922e-f050ff341750 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:33.310880 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1c1878-9919-4e0b-922e-f050ff341750 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:33.312245 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1c1878-9919-4e0b-922e-f050ff341750 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:33.324379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application [None req-fb1c1878-9919-4e0b-922e-f050ff341750 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 16:32:33.324379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:32:33.324379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 16:32:33.324379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 16:32:33.324379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 16:32:33.324379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 16:32:33.324379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 16:32:33.324379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:32:33.324379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 16:32:33.324379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:32:33.324379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 16:32:33.324379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:32:33.324379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 16:32:33.324379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 16:32:33.324379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 16:32:33.324379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 16:32:33.324379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 16:32:33.325577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 16:32:33.325577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 16:32:33.325577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 16:32:33.325577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 16:32:33.325577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 16:32:33.325577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 16:32:33.325577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 16:32:33.325577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 16:32:33.325577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 16:32:33.325577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 16:32:33.325577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 16:32:33.325577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 16:32:33.325577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 16:32:33.325577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 16:32:33.325577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 16:32:33.325577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.get_or_create( May 19 16:32:33.326448 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 16:32:33.326448 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application with Lock( May 19 16:32:33.326448 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 16:32:33.326448 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self._enter() May 19 16:32:33.326448 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 16:32:33.326448 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 16:32:33.326448 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 16:32:33.326448 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application return self.creator() May 19 16:32:33.326448 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 16:32:33.326448 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application created_value = creator( May 19 16:32:33.326448 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 16:32:33.326448 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 16:32:33.326448 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 16:32:33.326448 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: ERROR keystone.server.flask.application May 19 16:32:33.326448 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 642/1304] 10.222.0.36 () {56 vars in 1117 bytes} [Thu May 19 16:32:33 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 16:32:33.331665 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08fdcfc-a0e6-4991-a0b2-bab7ff3d1df3 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:33.407032 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:33.430451 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08fdcfc-a0e6-4991-a0b2-bab7ff3d1df3 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:33.463737 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b133bf-bd04-4e4a-8a84-3d3e96cbba5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:33.464399 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b133bf-bd04-4e4a-8a84-3d3e96cbba5e None admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:33.464684 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b133bf-bd04-4e4a-8a84-3d3e96cbba5e None admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:33.464964 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b133bf-bd04-4e4a-8a84-3d3e96cbba5e None admin] PATH_INFO: `/v3/services` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:33.465711 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b133bf-bd04-4e4a-8a84-3d3e96cbba5e None admin] RBAC: Adding query filter params (name=placement) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:32:33.466024 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b133bf-bd04-4e4a-8a84-3d3e96cbba5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:33.467428 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b133bf-bd04-4e4a-8a84-3d3e96cbba5e None admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:33.472638 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 663/1305] 10.222.0.36 () {56 vars in 1116 bytes} [Thu May 19 16:32:33 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:32:33.478360 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1c1878-9919-4e0b-922e-f050ff341750 None admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:32:33.575972 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1c1878-9919-4e0b-922e-f050ff341750 None admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:32:33.608865 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f167b5-3f5d-46ae-aeca-56c7973c5546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:32:33.609543 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f167b5-3f5d-46ae-aeca-56c7973c5546 None admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:32:33.609828 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f167b5-3f5d-46ae-aeca-56c7973c5546 None admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:32:33.610104 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f167b5-3f5d-46ae-aeca-56c7973c5546 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:32:33.610837 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f167b5-3f5d-46ae-aeca-56c7973c5546 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:32:33.612199 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f167b5-3f5d-46ae-aeca-56c7973c5546 None admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:32:33.630205 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 643/1306] 10.222.0.36 () {60 vars in 1144 bytes} [Thu May 19 16:32:33 2022] POST /identity/v3/endpoints => generated 331 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:32:43.469770 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=30 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:32:43.625537 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:33:45.577330 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04386c5-c754-4bab-b5fd-2217d45109fb None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:33:45.578188 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04386c5-c754-4bab-b5fd-2217d45109fb None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:33:45.578188 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04386c5-c754-4bab-b5fd-2217d45109fb None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:33:45.579969 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 664/1307] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:33:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:33:45.586360 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8fed2e-d8d0-47cd-8997-707d88772096 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:33:45.587181 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8fed2e-d8d0-47cd-8997-707d88772096 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:33:45.587181 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8fed2e-d8d0-47cd-8997-707d88772096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:33:45.735621 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-bc8fed2e-d8d0-47cd-8997-707d88772096 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:33:45.738138 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-bc8fed2e-d8d0-47cd-8997-707d88772096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:33:45.934850 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 644/1308] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:33:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:33:45.990999 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88862ff-aca4-40c2-a2b1-6f32041f4786 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:33:45.991450 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88862ff-aca4-40c2-a2b1-6f32041f4786 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:33:45.992287 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88862ff-aca4-40c2-a2b1-6f32041f4786 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:33:45.994014 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 665/1309] 10.222.0.36 () {54 vars in 964 bytes} [Thu May 19 16:33:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:33:46.003652 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e262aa78-3f70-4f04-8535-7df799fc001d None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:33:46.004116 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e262aa78-3f70-4f04-8535-7df799fc001d None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:33:46.004490 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e262aa78-3f70-4f04-8535-7df799fc001d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:33:46.153947 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-e262aa78-3f70-4f04-8535-7df799fc001d None None] MFA Rules not processed for user `33472643b65e4fe08fd5b3aa2c618133`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:33:46.156008 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-e262aa78-3f70-4f04-8535-7df799fc001d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:33:46.198901 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:33:46.308724 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 645/1310] 10.222.0.36 () {58 vars in 1061 bytes} [Thu May 19 16:33:46 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:33:46.325363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88862ff-aca4-40c2-a2b1-6f32041f4786 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:33:46.327236 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-f88862ff-aca4-40c2-a2b1-6f32041f4786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:33:46.544799 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88862ff-aca4-40c2-a2b1-6f32041f4786 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:33:46.703553 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b52251b-536b-4557-aa98-2e671cf3efa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:33:46.704289 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b52251b-536b-4557-aa98-2e671cf3efa7 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:33:46.704636 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b52251b-536b-4557-aa98-2e671cf3efa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:33:46.705219 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b52251b-536b-4557-aa98-2e671cf3efa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:33:46.708902 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-5b52251b-536b-4557-aa98-2e671cf3efa7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:33:46.807511 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:33:46.935093 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b52251b-536b-4557-aa98-2e671cf3efa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:33:46.938200 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b52251b-536b-4557-aa98-2e671cf3efa7 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:33:46.963319 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 666/1311] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:33:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 639 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 16:33:49.670743 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9296e09d-bc5f-465e-b639-abbd23ccbfb7 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:33:49.671319 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9296e09d-bc5f-465e-b639-abbd23ccbfb7 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:33:49.671715 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9296e09d-bc5f-465e-b639-abbd23ccbfb7 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:33:49.674939 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 646/1312] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:33:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:33:49.680669 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaaf58d-20cf-4408-a974-9ce216786834 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:33:49.681501 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaaf58d-20cf-4408-a974-9ce216786834 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:33:49.681501 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaaf58d-20cf-4408-a974-9ce216786834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:33:49.776185 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-ebaaf58d-20cf-4408-a974-9ce216786834 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:33:49.777496 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-ebaaf58d-20cf-4408-a974-9ce216786834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:33:49.899322 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 667/1313] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:33:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:33:49.951646 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc37027-d35c-4e51-a490-f7f3e66b1840 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:33:49.951937 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc37027-d35c-4e51-a490-f7f3e66b1840 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:33:49.952222 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc37027-d35c-4e51-a490-f7f3e66b1840 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:33:49.953923 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 647/1314] 10.222.0.36 () {54 vars in 964 bytes} [Thu May 19 16:33:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:33:49.960445 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad05e25-675a-4789-8d37-eea9efc02f6b None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:33:49.960739 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad05e25-675a-4789-8d37-eea9efc02f6b None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:33:49.961023 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad05e25-675a-4789-8d37-eea9efc02f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:33:49.972644 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:33:50.057844 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-2ad05e25-675a-4789-8d37-eea9efc02f6b None None] MFA Rules not processed for user `33472643b65e4fe08fd5b3aa2c618133`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:33:50.059086 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-2ad05e25-675a-4789-8d37-eea9efc02f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:33:50.164889 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 668/1315] 10.222.0.36 () {58 vars in 1061 bytes} [Thu May 19 16:33:49 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:33:50.175751 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc37027-d35c-4e51-a490-f7f3e66b1840 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:33:50.179199 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-0cc37027-d35c-4e51-a490-f7f3e66b1840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:33:50.319925 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc37027-d35c-4e51-a490-f7f3e66b1840 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:33:50.429292 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:33:50.433717 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9046447-de83-4c40-97d3-30a9782a86cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:33:50.436155 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9046447-de83-4c40-97d3-30a9782a86cc service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:33:50.437350 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9046447-de83-4c40-97d3-30a9782a86cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:33:50.437869 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9046447-de83-4c40-97d3-30a9782a86cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:33:50.441403 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-f9046447-de83-4c40-97d3-30a9782a86cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:33:50.627086 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9046447-de83-4c40-97d3-30a9782a86cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:33:50.628609 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9046447-de83-4c40-97d3-30a9782a86cc service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:33:50.651663 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 648/1316] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:33:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:33:53.815593 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419ebb1-1ad3-456c-8af4-2b863626d01f None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:33:53.816166 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419ebb1-1ad3-456c-8af4-2b863626d01f None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:33:53.816166 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419ebb1-1ad3-456c-8af4-2b863626d01f None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:33:53.817697 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 669/1317] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:33:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:33:53.826100 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a7a194-bde6-4430-a145-737caa8a6ab0 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:33:53.826656 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a7a194-bde6-4430-a145-737caa8a6ab0 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:33:53.826656 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a7a194-bde6-4430-a145-737caa8a6ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:33:53.904467 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-81a7a194-bde6-4430-a145-737caa8a6ab0 None None] MFA Rules not processed for user `a24c896dd82441dbb4ac8f0fab365a8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:33:53.906137 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-81a7a194-bde6-4430-a145-737caa8a6ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:33:53.969788 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:33:54.019763 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 649/1318] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:33:53 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:33:54.065912 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f419ebb1-1ad3-456c-8af4-2b863626d01f None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:33:54.068514 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-f419ebb1-1ad3-456c-8af4-2b863626d01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:33:54.206548 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f419ebb1-1ad3-456c-8af4-2b863626d01f None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:33:54.213278 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:33:54.302601 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468910c8-560c-4bd9-a066-fca407e8f49f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:33:54.303264 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468910c8-560c-4bd9-a066-fca407e8f49f service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:33:54.303582 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468910c8-560c-4bd9-a066-fca407e8f49f service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:33:54.303859 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468910c8-560c-4bd9-a066-fca407e8f49f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:33:54.306052 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-468910c8-560c-4bd9-a066-fca407e8f49f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:33:54.437142 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468910c8-560c-4bd9-a066-fca407e8f49f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:33:54.438889 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468910c8-560c-4bd9-a066-fca407e8f49f service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:33:54.463337 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 670/1319] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:33:54 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 16:33:57.815378 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5092b-acda-4b83-9172-2958b1bcedb1 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:33:57.816102 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5092b-acda-4b83-9172-2958b1bcedb1 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:33:57.816102 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5092b-acda-4b83-9172-2958b1bcedb1 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:33:57.817961 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 650/1320] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:33:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:33:57.824381 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef140ec-6d27-40b0-9926-d236ddb56282 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:33:57.825006 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef140ec-6d27-40b0-9926-d236ddb56282 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:33:57.825006 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef140ec-6d27-40b0-9926-d236ddb56282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:33:57.870117 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:33:57.906339 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-4ef140ec-6d27-40b0-9926-d236ddb56282 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:33:57.908467 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-4ef140ec-6d27-40b0-9926-d236ddb56282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:33:58.060607 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 671/1321] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:33:57 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:33:58.101226 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f5092b-acda-4b83-9172-2958b1bcedb1 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:33:58.102852 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-80f5092b-acda-4b83-9172-2958b1bcedb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:33:58.225503 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f5092b-acda-4b83-9172-2958b1bcedb1 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:33:58.313137 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb9acb2-b276-4ea2-a4e3-7856bb20d2db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:33:58.313832 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9acb2-b276-4ea2-a4e3-7856bb20d2db service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:33:58.314120 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9acb2-b276-4ea2-a4e3-7856bb20d2db service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:33:58.314401 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9acb2-b276-4ea2-a4e3-7856bb20d2db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:33:58.316155 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ebb9acb2-b276-4ea2-a4e3-7856bb20d2db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:33:58.324785 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:33:58.447454 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb9acb2-b276-4ea2-a4e3-7856bb20d2db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:33:58.448946 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb9acb2-b276-4ea2-a4e3-7856bb20d2db service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:33:58.471792 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 651/1322] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:33:58 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:34:01.728973 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c7c149-a36c-4411-80e8-410df254c452 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:34:01.729630 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c7c149-a36c-4411-80e8-410df254c452 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:34:01.729968 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c7c149-a36c-4411-80e8-410df254c452 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:34:01.732654 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 672/1323] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:34:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:34:01.739593 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b75e4-a232-4516-b4f3-92f19e4f78c0 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:34:01.740398 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b75e4-a232-4516-b4f3-92f19e4f78c0 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:34:01.740398 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b75e4-a232-4516-b4f3-92f19e4f78c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:34:01.823612 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-e84b75e4-a232-4516-b4f3-92f19e4f78c0 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:34:01.824879 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-e84b75e4-a232-4516-b4f3-92f19e4f78c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:34:01.913576 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:34:01.977782 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 652/1324] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:34:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:34:02.041198 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c7c149-a36c-4411-80e8-410df254c452 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:34:02.042925 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-04c7c149-a36c-4411-80e8-410df254c452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:34:02.208854 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c7c149-a36c-4411-80e8-410df254c452 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:34:02.229527 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:34:02.308635 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169c4a21-2019-4d32-8c89-4255dea5c9d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:34:02.309222 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c4a21-2019-4d32-8c89-4255dea5c9d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:34:02.309497 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c4a21-2019-4d32-8c89-4255dea5c9d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:34:02.309791 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c4a21-2019-4d32-8c89-4255dea5c9d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:34:02.311623 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-169c4a21-2019-4d32-8c89-4255dea5c9d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:34:02.458568 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169c4a21-2019-4d32-8c89-4255dea5c9d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:34:02.460081 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169c4a21-2019-4d32-8c89-4255dea5c9d5 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:34:02.483600 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 673/1325] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:34:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:34:04.941699 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2320793-856d-4ade-81e1-f89ea30c9abb None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:34:04.942570 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2320793-856d-4ade-81e1-f89ea30c9abb None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:34:04.942570 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2320793-856d-4ade-81e1-f89ea30c9abb None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:34:04.943730 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 653/1326] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:34:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:34:04.950195 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6845f3a-7597-4ac5-b168-a42d51c230f9 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:34:04.951042 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6845f3a-7597-4ac5-b168-a42d51c230f9 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:34:04.951042 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6845f3a-7597-4ac5-b168-a42d51c230f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:34:05.011786 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:34:05.035573 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-f6845f3a-7597-4ac5-b168-a42d51c230f9 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:34:05.037416 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-f6845f3a-7597-4ac5-b168-a42d51c230f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:34:05.174118 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 674/1327] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:34:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:34:05.213798 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2320793-856d-4ade-81e1-f89ea30c9abb None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:34:05.215416 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-f2320793-856d-4ade-81e1-f89ea30c9abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:34:05.356987 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2320793-856d-4ade-81e1-f89ea30c9abb None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:34:05.462525 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0032e18d-db74-429b-957e-ba8fb37d5bc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:34:05.463109 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0032e18d-db74-429b-957e-ba8fb37d5bc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:34:05.463401 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0032e18d-db74-429b-957e-ba8fb37d5bc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:34:05.463689 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0032e18d-db74-429b-957e-ba8fb37d5bc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:34:05.465481 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-0032e18d-db74-429b-957e-ba8fb37d5bc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:34:05.484843 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:34:05.641634 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0032e18d-db74-429b-957e-ba8fb37d5bc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:34:05.644571 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0032e18d-db74-429b-957e-ba8fb37d5bc9 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:34:05.682789 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 654/1328] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:34:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:34:08.123200 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ffba0-2f84-4e5e-85b9-1ba7e62b084a None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:34:08.123989 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ffba0-2f84-4e5e-85b9-1ba7e62b084a None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:34:08.124150 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ffba0-2f84-4e5e-85b9-1ba7e62b084a None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:34:08.125921 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 675/1329] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:34:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:34:08.132475 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de581e-d3fb-44ec-a0b7-8dd69802a2d6 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:34:08.133085 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de581e-d3fb-44ec-a0b7-8dd69802a2d6 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:34:08.133085 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de581e-d3fb-44ec-a0b7-8dd69802a2d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:34:08.229589 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-e5de581e-d3fb-44ec-a0b7-8dd69802a2d6 None None] MFA Rules not processed for user `a24c896dd82441dbb4ac8f0fab365a8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:34:08.231725 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-e5de581e-d3fb-44ec-a0b7-8dd69802a2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:34:08.314473 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:34:08.355639 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 655/1330] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:34:08 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:34:08.402723 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417ffba0-2f84-4e5e-85b9-1ba7e62b084a None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:34:08.404994 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-417ffba0-2f84-4e5e-85b9-1ba7e62b084a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:34:08.542605 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417ffba0-2f84-4e5e-85b9-1ba7e62b084a None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:34:08.568519 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:34:08.639394 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aba865-80d8-40f4-9388-19fc942552f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:34:08.640072 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aba865-80d8-40f4-9388-19fc942552f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:34:08.640364 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aba865-80d8-40f4-9388-19fc942552f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:34:08.640647 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aba865-80d8-40f4-9388-19fc942552f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:34:08.642616 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d1aba865-80d8-40f4-9388-19fc942552f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:34:08.793535 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aba865-80d8-40f4-9388-19fc942552f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:34:08.795117 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aba865-80d8-40f4-9388-19fc942552f2 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:34:08.817665 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 676/1331] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:34:08 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 16:34:11.472639 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82581b8b-d6b3-4c3a-ac7d-c035a5348c9d None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:34:11.473252 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82581b8b-d6b3-4c3a-ac7d-c035a5348c9d None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:34:11.474267 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82581b8b-d6b3-4c3a-ac7d-c035a5348c9d None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:34:11.477024 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 656/1332] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:34:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:34:11.484004 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d313dcc6-465e-4880-ad4a-74bf24c4a6ce None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:34:11.484695 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d313dcc6-465e-4880-ad4a-74bf24c4a6ce None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:34:11.485312 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d313dcc6-465e-4880-ad4a-74bf24c4a6ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:34:11.602688 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:34:11.608468 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-d313dcc6-465e-4880-ad4a-74bf24c4a6ce None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:34:11.610873 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d313dcc6-465e-4880-ad4a-74bf24c4a6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:34:11.771737 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 677/1333] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:34:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:34:11.810470 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82581b8b-d6b3-4c3a-ac7d-c035a5348c9d None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:34:11.812068 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-82581b8b-d6b3-4c3a-ac7d-c035a5348c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:34:11.943096 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82581b8b-d6b3-4c3a-ac7d-c035a5348c9d None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:34:12.068032 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1375cf7-3af6-48c7-a8d2-85cc5eda6896 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:34:12.069143 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1375cf7-3af6-48c7-a8d2-85cc5eda6896 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:34:12.069708 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1375cf7-3af6-48c7-a8d2-85cc5eda6896 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:34:12.070211 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1375cf7-3af6-48c7-a8d2-85cc5eda6896 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:34:12.073569 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-a1375cf7-3af6-48c7-a8d2-85cc5eda6896 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:34:12.119127 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:34:12.248174 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1375cf7-3af6-48c7-a8d2-85cc5eda6896 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:34:12.250812 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1375cf7-3af6-48c7-a8d2-85cc5eda6896 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:34:12.289274 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 657/1334] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:34:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:34:14.845821 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bda6c8-78b8-46ba-b988-3f345707a439 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:34:14.846433 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bda6c8-78b8-46ba-b988-3f345707a439 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:34:14.846588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bda6c8-78b8-46ba-b988-3f345707a439 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:34:14.847898 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 678/1335] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:34:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:34:14.854261 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00315bf-ee1d-4405-bfdb-ab8d7de7ee8f None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:34:14.854814 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00315bf-ee1d-4405-bfdb-ab8d7de7ee8f None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:34:14.855059 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00315bf-ee1d-4405-bfdb-ab8d7de7ee8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:34:14.944016 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-c00315bf-ee1d-4405-bfdb-ab8d7de7ee8f None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:34:14.945588 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-c00315bf-ee1d-4405-bfdb-ab8d7de7ee8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:34:15.034401 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:34:15.060817 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 658/1336] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:34:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:34:15.103629 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bda6c8-78b8-46ba-b988-3f345707a439 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:34:15.105224 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-60bda6c8-78b8-46ba-b988-3f345707a439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:34:15.250931 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bda6c8-78b8-46ba-b988-3f345707a439 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:34:15.283191 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:34:15.333440 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25ab6c3-27ec-4bcc-b214-6e020e990d95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:34:15.334267 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25ab6c3-27ec-4bcc-b214-6e020e990d95 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:34:15.334559 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25ab6c3-27ec-4bcc-b214-6e020e990d95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:34:15.334842 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25ab6c3-27ec-4bcc-b214-6e020e990d95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:34:15.336689 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-f25ab6c3-27ec-4bcc-b214-6e020e990d95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:34:15.456708 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25ab6c3-27ec-4bcc-b214-6e020e990d95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:34:15.458871 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25ab6c3-27ec-4bcc-b214-6e020e990d95 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:34:15.480776 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 679/1337] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:34:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:34:18.339057 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb218c-6215-45f9-948e-62e1189b93dd None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:34:18.339579 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb218c-6215-45f9-948e-62e1189b93dd None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:34:18.339579 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb218c-6215-45f9-948e-62e1189b93dd None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:34:18.340913 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 659/1338] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:34:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:34:18.346577 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c57fa4d-da7b-4cd3-b5cd-065b2bdee4fa None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:34:18.347094 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c57fa4d-da7b-4cd3-b5cd-065b2bdee4fa None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:34:18.347190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c57fa4d-da7b-4cd3-b5cd-065b2bdee4fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:34:18.466302 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-0c57fa4d-da7b-4cd3-b5cd-065b2bdee4fa None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:34:18.467545 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-0c57fa4d-da7b-4cd3-b5cd-065b2bdee4fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:34:18.477764 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:34:18.585905 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 680/1339] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:34:18 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:34:18.622526 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bb218c-6215-45f9-948e-62e1189b93dd None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:34:18.625019 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-06bb218c-6215-45f9-948e-62e1189b93dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:34:18.748111 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bb218c-6215-45f9-948e-62e1189b93dd None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:34:18.752622 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3517e4db-455a-4383-9923-0b18f88c2bae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:34:18.753476 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3517e4db-455a-4383-9923-0b18f88c2bae service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:34:18.753773 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3517e4db-455a-4383-9923-0b18f88c2bae service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:34:18.754056 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3517e4db-455a-4383-9923-0b18f88c2bae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:34:18.764263 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3517e4db-455a-4383-9923-0b18f88c2bae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:34:18.765771 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3517e4db-455a-4383-9923-0b18f88c2bae service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:34:18.775884 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 660/1340] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:34:18 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:34:23.531037 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774e59b-a6e9-490e-9575-07d491eb7985 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:34:23.531857 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774e59b-a6e9-490e-9575-07d491eb7985 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:34:23.532345 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774e59b-a6e9-490e-9575-07d491eb7985 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:34:23.545061 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 681/1341] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:34:23 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 16:34:23.555342 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6bcf6e-9f8c-41f3-8967-b39f60917412 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:34:23.556211 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6bcf6e-9f8c-41f3-8967-b39f60917412 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:34:23.556398 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6bcf6e-9f8c-41f3-8967-b39f60917412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:34:23.609047 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-fe6bcf6e-9f8c-41f3-8967-b39f60917412 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:34:23.611231 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-fe6bcf6e-9f8c-41f3-8967-b39f60917412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:34:23.709392 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 661/1342] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:34:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:34:23.751165 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e774e59b-a6e9-490e-9575-07d491eb7985 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:34:23.762887 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e774e59b-a6e9-490e-9575-07d491eb7985 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:34:23.767207 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11df5cd-e6ba-466b-9060-82bc6fcfcf8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:34:23.767992 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11df5cd-e6ba-466b-9060-82bc6fcfcf8f service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:34:23.768344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11df5cd-e6ba-466b-9060-82bc6fcfcf8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:34:23.768635 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11df5cd-e6ba-466b-9060-82bc6fcfcf8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:34:23.779094 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11df5cd-e6ba-466b-9060-82bc6fcfcf8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:34:23.780587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11df5cd-e6ba-466b-9060-82bc6fcfcf8f service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:34:23.792355 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 682/1343] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:34:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 16:34:27.086267 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc34d2d-045c-4204-86f5-07c68c3d1184 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:34:27.087102 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc34d2d-045c-4204-86f5-07c68c3d1184 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:34:27.087102 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc34d2d-045c-4204-86f5-07c68c3d1184 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:34:27.088224 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 662/1344] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:34:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 16:34:27.094890 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a15e3-975d-452a-ba8b-e0f88f334f58 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:34:27.095461 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a15e3-975d-452a-ba8b-e0f88f334f58 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:34:27.096056 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a15e3-975d-452a-ba8b-e0f88f334f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:34:27.130477 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-0b1a15e3-975d-452a-ba8b-e0f88f334f58 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:34:27.132609 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-0b1a15e3-975d-452a-ba8b-e0f88f334f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:34:27.147350 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 683/1345] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:34:27 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:34:27.188150 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc34d2d-045c-4204-86f5-07c68c3d1184 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:34:27.190015 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-5dc34d2d-045c-4204-86f5-07c68c3d1184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:34:27.204916 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc34d2d-045c-4204-86f5-07c68c3d1184 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:34:27.208818 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1468e2-d3b9-49e1-ad03-e1bbfa6626db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:34:27.209522 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1468e2-d3b9-49e1-ad03-e1bbfa6626db service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:34:27.209807 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1468e2-d3b9-49e1-ad03-e1bbfa6626db service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:34:27.210084 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1468e2-d3b9-49e1-ad03-e1bbfa6626db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:34:27.220921 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1468e2-d3b9-49e1-ad03-e1bbfa6626db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:34:27.222450 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1468e2-d3b9-49e1-ad03-e1bbfa6626db service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:34:27.233684 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 663/1346] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:34:27 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:34:32.688549 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9595f-adbc-4812-8eba-0c68faeadc01 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:34:32.689457 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9595f-adbc-4812-8eba-0c68faeadc01 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:34:32.689457 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9595f-adbc-4812-8eba-0c68faeadc01 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:34:32.691000 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 684/1347] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:34:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 16:34:32.697289 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2ec67-6b8c-4005-8107-509001cbfe67 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:34:32.698282 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2ec67-6b8c-4005-8107-509001cbfe67 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:34:32.698282 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2ec67-6b8c-4005-8107-509001cbfe67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:34:32.719356 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-57a2ec67-6b8c-4005-8107-509001cbfe67 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:34:32.720661 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-57a2ec67-6b8c-4005-8107-509001cbfe67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:34:32.730278 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 664/1348] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:34:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:34:32.783638 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd9595f-adbc-4812-8eba-0c68faeadc01 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:34:32.796814 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd9595f-adbc-4812-8eba-0c68faeadc01 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:34:32.801034 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f013246-0b92-4ee1-9ab2-1a7f89fda0d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:34:32.802608 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f013246-0b92-4ee1-9ab2-1a7f89fda0d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:34:32.802959 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f013246-0b92-4ee1-9ab2-1a7f89fda0d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:34:32.803326 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f013246-0b92-4ee1-9ab2-1a7f89fda0d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:34:32.814607 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f013246-0b92-4ee1-9ab2-1a7f89fda0d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:34:32.816180 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f013246-0b92-4ee1-9ab2-1a7f89fda0d6 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:34:32.828778 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 685/1349] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:34:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:34:40.057388 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba90cc8-5f1f-4b12-843b-b506aff28f68 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:34:40.058237 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba90cc8-5f1f-4b12-843b-b506aff28f68 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:34:40.058668 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba90cc8-5f1f-4b12-843b-b506aff28f68 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:34:40.060576 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 665/1350] 10.222.0.36 () {54 vars in 921 bytes} [Thu May 19 16:34:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:34:40.068094 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47272d5-cfdb-4c3d-8242-7a830944ff05 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:34:40.069766 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47272d5-cfdb-4c3d-8242-7a830944ff05 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:34:40.070501 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47272d5-cfdb-4c3d-8242-7a830944ff05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:34:40.235154 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-b47272d5-cfdb-4c3d-8242-7a830944ff05 None None] MFA Rules not processed for user `c6e158e5c87f4954aeb297f42dabdc16`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:34:40.237895 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b47272d5-cfdb-4c3d-8242-7a830944ff05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:34:40.405419 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:34:40.480151 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 686/1351] 10.222.0.36 () {58 vars in 1018 bytes} [Thu May 19 16:34:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 414 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:34:42.712730 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=98 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:34:45.379476 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b09a3-cf98-412e-8609-2b1d83d6e56c None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:34:45.380468 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b09a3-cf98-412e-8609-2b1d83d6e56c None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:34:45.380468 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b09a3-cf98-412e-8609-2b1d83d6e56c None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:34:45.381895 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 666/1352] 10.222.0.36 () {54 vars in 921 bytes} [Thu May 19 16:34:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:34:45.388440 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef881f-3794-4629-8491-7753347668e7 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:34:45.388980 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef881f-3794-4629-8491-7753347668e7 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:34:45.389708 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef881f-3794-4629-8491-7753347668e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:34:45.421801 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-eeef881f-3794-4629-8491-7753347668e7 None None] MFA Rules not processed for user `c6e158e5c87f4954aeb297f42dabdc16`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:34:45.424346 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-eeef881f-3794-4629-8491-7753347668e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:34:45.444082 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 687/1353] 10.222.0.36 () {58 vars in 1018 bytes} [Thu May 19 16:34:45 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:34:47.689336 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5db91d-2883-4ae4-882a-1e551346992f None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:34:47.690180 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5db91d-2883-4ae4-882a-1e551346992f None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:34:47.690180 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5db91d-2883-4ae4-882a-1e551346992f None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:34:47.692172 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 667/1354] 10.222.0.36 () {54 vars in 921 bytes} [Thu May 19 16:34:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 16:34:47.698505 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb97ca9-63c0-4598-b224-698f46fa8c5d None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:34:47.699215 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb97ca9-63c0-4598-b224-698f46fa8c5d None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:34:47.699502 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb97ca9-63c0-4598-b224-698f46fa8c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:34:47.726804 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-1fb97ca9-63c0-4598-b224-698f46fa8c5d None None] MFA Rules not processed for user `c6e158e5c87f4954aeb297f42dabdc16`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:34:47.728288 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-1fb97ca9-63c0-4598-b224-698f46fa8c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:34:47.737735 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 688/1355] 10.222.0.36 () {58 vars in 1018 bytes} [Thu May 19 16:34:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:34:52.546389 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0531acc0-ea94-4bec-b02d-03de70548f30 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:34:52.546914 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0531acc0-ea94-4bec-b02d-03de70548f30 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:34:52.547275 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0531acc0-ea94-4bec-b02d-03de70548f30 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:34:52.550189 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 668/1356] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:34:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:34:52.557277 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b84b11-177b-429e-a264-08eea388bc99 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:34:52.558164 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b84b11-177b-429e-a264-08eea388bc99 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:34:52.558456 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b84b11-177b-429e-a264-08eea388bc99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:34:52.580521 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-38b84b11-177b-429e-a264-08eea388bc99 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:34:52.583764 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-38b84b11-177b-429e-a264-08eea388bc99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:34:52.601221 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 689/1357] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:34:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:34:52.672167 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09df95-2d68-45ed-b573-9f2482b3d08b None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:34:52.672787 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09df95-2d68-45ed-b573-9f2482b3d08b None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:34:52.673335 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09df95-2d68-45ed-b573-9f2482b3d08b None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:34:52.675131 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 669/1358] 10.222.0.36 () {54 vars in 961 bytes} [Thu May 19 16:34:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:34:52.688262 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c94fbe-5ff0-4aa8-b306-3443f2e9f0ac None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:34:52.690498 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c94fbe-5ff0-4aa8-b306-3443f2e9f0ac None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:34:52.691337 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c94fbe-5ff0-4aa8-b306-3443f2e9f0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:34:52.765780 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-a8c94fbe-5ff0-4aa8-b306-3443f2e9f0ac None None] MFA Rules not processed for user `58bdeada00b74b7b8e186e6dc51e6dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:34:52.767400 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-a8c94fbe-5ff0-4aa8-b306-3443f2e9f0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:34:52.890662 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 690/1359] 10.222.0.36 () {58 vars in 1058 bytes} [Thu May 19 16:34:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:34:52.902842 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a09df95-2d68-45ed-b573-9f2482b3d08b None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:34:52.932372 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a09df95-2d68-45ed-b573-9f2482b3d08b None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:34:52.943291 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee6e7ad-7024-4d3d-80ab-1f05b1846043 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:34:52.944612 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee6e7ad-7024-4d3d-80ab-1f05b1846043 service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:34:52.945226 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee6e7ad-7024-4d3d-80ab-1f05b1846043 service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:34:52.945813 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee6e7ad-7024-4d3d-80ab-1f05b1846043 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:34:52.964141 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee6e7ad-7024-4d3d-80ab-1f05b1846043 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:34:52.967628 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee6e7ad-7024-4d3d-80ab-1f05b1846043 service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:34:52.986550 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 670/1360] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:34:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:34:56.436890 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435da0e7-7fa7-4239-b26d-b7d76aaea4fc None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:34:56.437809 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435da0e7-7fa7-4239-b26d-b7d76aaea4fc None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:34:56.437958 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435da0e7-7fa7-4239-b26d-b7d76aaea4fc None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:34:56.440672 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 691/1361] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:34:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:34:56.451343 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133208f0-9342-44f7-92b9-5f7110737fa3 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:34:56.453024 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133208f0-9342-44f7-92b9-5f7110737fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:34:56.453760 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133208f0-9342-44f7-92b9-5f7110737fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:34:56.485780 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-133208f0-9342-44f7-92b9-5f7110737fa3 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:34:56.487981 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-133208f0-9342-44f7-92b9-5f7110737fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:34:56.503759 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 671/1362] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:34:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:34:56.589525 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435da0e7-7fa7-4239-b26d-b7d76aaea4fc None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:34:56.612835 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435da0e7-7fa7-4239-b26d-b7d76aaea4fc None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:34:56.624142 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8529a90c-d5ce-4c55-ac76-cd20822e0ab6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:34:56.625695 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8529a90c-d5ce-4c55-ac76-cd20822e0ab6 service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:34:56.626120 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8529a90c-d5ce-4c55-ac76-cd20822e0ab6 service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:34:56.626616 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8529a90c-d5ce-4c55-ac76-cd20822e0ab6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:34:56.644736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8529a90c-d5ce-4c55-ac76-cd20822e0ab6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:34:56.648627 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8529a90c-d5ce-4c55-ac76-cd20822e0ab6 service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:34:56.670332 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 692/1363] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:34:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:35:00.769354 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef98240-5540-477f-aba6-bb59accd3339 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:00.770306 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef98240-5540-477f-aba6-bb59accd3339 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:00.770618 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef98240-5540-477f-aba6-bb59accd3339 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:00.772518 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 672/1364] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:35:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:35:00.782537 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a51c4a-8000-49d8-829e-1d1cb74e145a None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:00.784005 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a51c4a-8000-49d8-829e-1d1cb74e145a None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:00.784515 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a51c4a-8000-49d8-829e-1d1cb74e145a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:00.816259 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-54a51c4a-8000-49d8-829e-1d1cb74e145a None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:35:00.819058 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-54a51c4a-8000-49d8-829e-1d1cb74e145a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:35:00.833205 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 693/1365] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:35:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:35:00.918541 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da8077a-15b8-4e9e-8cee-5bde5158f42d None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:00.919288 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da8077a-15b8-4e9e-8cee-5bde5158f42d None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:00.919676 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da8077a-15b8-4e9e-8cee-5bde5158f42d None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:00.921667 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 673/1366] 10.222.0.36 () {54 vars in 961 bytes} [Thu May 19 16:35:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:35:00.935976 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f1be8-46eb-4e6a-bc11-c5bcddee414d None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:00.937071 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f1be8-46eb-4e6a-bc11-c5bcddee414d None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:00.937704 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f1be8-46eb-4e6a-bc11-c5bcddee414d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:00.970533 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-f37f1be8-46eb-4e6a-bc11-c5bcddee414d None None] MFA Rules not processed for user `58bdeada00b74b7b8e186e6dc51e6dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:35:00.972940 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-f37f1be8-46eb-4e6a-bc11-c5bcddee414d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:35:00.991316 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 694/1367] 10.222.0.36 () {58 vars in 1058 bytes} [Thu May 19 16:35:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:35:01.000999 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da8077a-15b8-4e9e-8cee-5bde5158f42d None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:35:01.022301 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da8077a-15b8-4e9e-8cee-5bde5158f42d None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:35:01.030940 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3c3d04-248f-4674-964d-e912d93170d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:35:01.031875 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c3d04-248f-4674-964d-e912d93170d9 service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:01.032357 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c3d04-248f-4674-964d-e912d93170d9 service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:01.032895 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c3d04-248f-4674-964d-e912d93170d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:01.045075 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3c3d04-248f-4674-964d-e912d93170d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:35:01.047157 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3c3d04-248f-4674-964d-e912d93170d9 service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:35:01.061005 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 674/1368] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:35:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 16:35:02.540661 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24cc3fd-d560-4550-b981-07099bc3372a None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:02.541685 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24cc3fd-d560-4550-b981-07099bc3372a None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:02.542185 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24cc3fd-d560-4550-b981-07099bc3372a None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:02.543931 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 695/1369] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:35:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 16:35:02.550181 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2a85e-630a-423b-a4b5-3392f676f96c None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:02.551624 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2a85e-630a-423b-a4b5-3392f676f96c None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:02.551624 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2a85e-630a-423b-a4b5-3392f676f96c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:02.571916 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-dbe2a85e-630a-423b-a4b5-3392f676f96c None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:35:02.573381 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-dbe2a85e-630a-423b-a4b5-3392f676f96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:35:02.588440 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 675/1370] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:35:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:35:02.909605 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbbee29-8534-4283-9157-4dccff1f8557 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:02.909924 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbbee29-8534-4283-9157-4dccff1f8557 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:02.910083 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbbee29-8534-4283-9157-4dccff1f8557 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:02.913389 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 696/1371] 10.222.0.36 () {54 vars in 963 bytes} [Thu May 19 16:35:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:35:02.940685 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242637c-8089-4289-8010-8f7bbe91deca None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:02.941554 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242637c-8089-4289-8010-8f7bbe91deca None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:02.942355 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242637c-8089-4289-8010-8f7bbe91deca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:03.048058 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-1242637c-8089-4289-8010-8f7bbe91deca None None] MFA Rules not processed for user `2cc96d859a344042a4251a4104419855`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:35:03.051391 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-1242637c-8089-4289-8010-8f7bbe91deca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:35:03.191419 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 676/1372] 10.222.0.36 () {58 vars in 1060 bytes} [Thu May 19 16:35:02 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:35:03.198004 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbbee29-8534-4283-9157-4dccff1f8557 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:35:03.221534 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbbee29-8534-4283-9157-4dccff1f8557 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:35:03.228824 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10cf259-d5d9-4fcf-beb4-ea8e5ddf1e59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:35:03.230723 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cf259-d5d9-4fcf-beb4-ea8e5ddf1e59 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:03.231925 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cf259-d5d9-4fcf-beb4-ea8e5ddf1e59 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:03.232746 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cf259-d5d9-4fcf-beb4-ea8e5ddf1e59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:03.253095 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10cf259-d5d9-4fcf-beb4-ea8e5ddf1e59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:35:03.255272 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10cf259-d5d9-4fcf-beb4-ea8e5ddf1e59 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:35:03.278537 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 697/1373] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:35:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:35:04.298508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd5c8a-637c-47c7-b616-b02935a4bcee None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:04.299536 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd5c8a-637c-47c7-b616-b02935a4bcee None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:04.299536 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd5c8a-637c-47c7-b616-b02935a4bcee None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:04.302547 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 677/1374] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:35:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 16:35:04.308007 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0870be-bea4-427d-96d0-a58f17cc784d None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:04.308770 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0870be-bea4-427d-96d0-a58f17cc784d None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:04.308770 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0870be-bea4-427d-96d0-a58f17cc784d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:04.338785 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-4d0870be-bea4-427d-96d0-a58f17cc784d None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:35:04.342848 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-4d0870be-bea4-427d-96d0-a58f17cc784d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:35:04.358459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 698/1375] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:35:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:35:04.399604 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd5c8a-637c-47c7-b616-b02935a4bcee None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:35:04.427118 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd5c8a-637c-47c7-b616-b02935a4bcee None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:35:04.436897 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c5737-0722-4673-8a35-015dacc0534b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:35:04.440954 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c5737-0722-4673-8a35-015dacc0534b service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:04.441429 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c5737-0722-4673-8a35-015dacc0534b service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:04.441799 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c5737-0722-4673-8a35-015dacc0534b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:04.455113 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c5737-0722-4673-8a35-015dacc0534b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:35:04.456858 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c5737-0722-4673-8a35-015dacc0534b service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:35:04.468284 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 678/1376] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:35:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:35:07.058825 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283493e8-0987-4d94-a0bf-4a416f6950de None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:07.059587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283493e8-0987-4d94-a0bf-4a416f6950de None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:07.059587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283493e8-0987-4d94-a0bf-4a416f6950de None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:07.061082 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 699/1377] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:35:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:35:07.069025 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8dc878-bfe9-4479-9234-60b1b1c214e0 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:07.069834 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8dc878-bfe9-4479-9234-60b1b1c214e0 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:07.070050 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8dc878-bfe9-4479-9234-60b1b1c214e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:07.094788 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-cd8dc878-bfe9-4479-9234-60b1b1c214e0 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:35:07.097120 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-cd8dc878-bfe9-4479-9234-60b1b1c214e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:35:07.116915 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 679/1378] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:35:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:35:07.164320 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283493e8-0987-4d94-a0bf-4a416f6950de None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:35:07.180924 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283493e8-0987-4d94-a0bf-4a416f6950de None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:35:07.187310 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f51f81-830b-43c2-87a4-5c633f188cfe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:35:07.188050 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f51f81-830b-43c2-87a4-5c633f188cfe service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:07.188417 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f51f81-830b-43c2-87a4-5c633f188cfe service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:07.188853 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f51f81-830b-43c2-87a4-5c633f188cfe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:07.205210 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f51f81-830b-43c2-87a4-5c633f188cfe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:35:07.205210 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f51f81-830b-43c2-87a4-5c633f188cfe service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:35:07.222313 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 700/1379] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:35:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 16:35:10.112649 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b2250c-4507-4cea-8870-1c2ca5b1f864 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:10.114237 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b2250c-4507-4cea-8870-1c2ca5b1f864 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:10.114684 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b2250c-4507-4cea-8870-1c2ca5b1f864 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:10.121590 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 680/1380] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:35:10 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:35:10.124082 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232d16ae-48f2-4f21-b081-e93b83fc329f None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:10.125234 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232d16ae-48f2-4f21-b081-e93b83fc329f None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:10.125740 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232d16ae-48f2-4f21-b081-e93b83fc329f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:10.146472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-232d16ae-48f2-4f21-b081-e93b83fc329f None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:35:10.148138 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-232d16ae-48f2-4f21-b081-e93b83fc329f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:35:10.157101 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 701/1381] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:35:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:35:10.237855 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b2250c-4507-4cea-8870-1c2ca5b1f864 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:35:10.253955 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b2250c-4507-4cea-8870-1c2ca5b1f864 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:35:10.259984 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d980394c-ea6d-49dd-91cc-80cc1ef3e48f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:35:10.260648 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980394c-ea6d-49dd-91cc-80cc1ef3e48f service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:10.260930 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980394c-ea6d-49dd-91cc-80cc1ef3e48f service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:10.261237 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980394c-ea6d-49dd-91cc-80cc1ef3e48f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:10.271177 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d980394c-ea6d-49dd-91cc-80cc1ef3e48f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:35:10.272817 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d980394c-ea6d-49dd-91cc-80cc1ef3e48f service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:35:10.283935 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 681/1382] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:35:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 16:35:12.713077 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e5cd4-e4b3-47a2-90cc-9fb0df652b1c None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:12.713696 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e5cd4-e4b3-47a2-90cc-9fb0df652b1c None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:12.713969 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e5cd4-e4b3-47a2-90cc-9fb0df652b1c None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:12.715520 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 702/1383] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:35:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:35:12.721653 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab68296-6e7b-4a70-a406-8d6c3db611a3 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:12.722192 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab68296-6e7b-4a70-a406-8d6c3db611a3 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:12.722192 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab68296-6e7b-4a70-a406-8d6c3db611a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:12.746141 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-aab68296-6e7b-4a70-a406-8d6c3db611a3 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:35:12.747399 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-aab68296-6e7b-4a70-a406-8d6c3db611a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:35:12.757344 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 682/1384] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:35:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:35:12.830806 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8e5cd4-e4b3-47a2-90cc-9fb0df652b1c None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:35:12.841246 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=95 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:35:12.851222 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8e5cd4-e4b3-47a2-90cc-9fb0df652b1c None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:35:12.858419 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae0d3c-83d1-49c0-8b5b-adaaf7ffb063 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:35:12.859909 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae0d3c-83d1-49c0-8b5b-adaaf7ffb063 service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:12.860416 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae0d3c-83d1-49c0-8b5b-adaaf7ffb063 service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:12.860901 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae0d3c-83d1-49c0-8b5b-adaaf7ffb063 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:12.874392 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ae0d3c-83d1-49c0-8b5b-adaaf7ffb063 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:35:12.877732 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ae0d3c-83d1-49c0-8b5b-adaaf7ffb063 service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:35:12.900243 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 703/1385] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:35:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:35:15.729056 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78022a3e-de2d-4108-8561-522e12e31ab8 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:15.729823 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78022a3e-de2d-4108-8561-522e12e31ab8 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:15.730017 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78022a3e-de2d-4108-8561-522e12e31ab8 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:15.731786 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 683/1386] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:35:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:35:15.740619 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d03a33-b378-47ff-87cb-788d2f448ef0 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:15.741458 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d03a33-b378-47ff-87cb-788d2f448ef0 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:15.741617 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d03a33-b378-47ff-87cb-788d2f448ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:15.764562 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-e1d03a33-b378-47ff-87cb-788d2f448ef0 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:35:15.766253 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e1d03a33-b378-47ff-87cb-788d2f448ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:35:15.777136 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 704/1387] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:35:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:35:15.857791 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78022a3e-de2d-4108-8561-522e12e31ab8 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:35:15.881802 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78022a3e-de2d-4108-8561-522e12e31ab8 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:35:15.891960 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7060895f-da25-45a8-a663-bafc4f1ab1fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:35:15.892990 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060895f-da25-45a8-a663-bafc4f1ab1fb service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:15.893301 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060895f-da25-45a8-a663-bafc4f1ab1fb service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:15.893572 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060895f-da25-45a8-a663-bafc4f1ab1fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:15.907868 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7060895f-da25-45a8-a663-bafc4f1ab1fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:35:15.910100 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7060895f-da25-45a8-a663-bafc4f1ab1fb service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:35:15.926917 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 684/1388] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:35:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 16:35:18.597453 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e43460-99af-4a68-8512-d60ffd474267 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:18.598275 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e43460-99af-4a68-8512-d60ffd474267 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:18.598275 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e43460-99af-4a68-8512-d60ffd474267 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:18.601036 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 705/1389] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:35:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:35:18.609985 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba069709-7eca-4e52-aaf9-601dd0a15bcc None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:18.610721 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba069709-7eca-4e52-aaf9-601dd0a15bcc None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:18.611128 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba069709-7eca-4e52-aaf9-601dd0a15bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:18.638524 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-ba069709-7eca-4e52-aaf9-601dd0a15bcc None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:35:18.640443 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ba069709-7eca-4e52-aaf9-601dd0a15bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:35:18.653136 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 685/1390] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:35:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:35:18.722186 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e43460-99af-4a68-8512-d60ffd474267 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:35:18.743142 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e43460-99af-4a68-8512-d60ffd474267 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:35:18.760147 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06970df6-bb23-4385-afa0-1dff2ce6f244 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:35:18.761328 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06970df6-bb23-4385-afa0-1dff2ce6f244 service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:18.761831 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06970df6-bb23-4385-afa0-1dff2ce6f244 service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:18.762334 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06970df6-bb23-4385-afa0-1dff2ce6f244 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:18.781499 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06970df6-bb23-4385-afa0-1dff2ce6f244 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:35:18.785743 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06970df6-bb23-4385-afa0-1dff2ce6f244 service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:35:18.808276 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 706/1391] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:35:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:35:21.329063 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664a0751-3552-4882-ae5d-460fe451e3a8 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:21.329699 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664a0751-3552-4882-ae5d-460fe451e3a8 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:21.329699 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664a0751-3552-4882-ae5d-460fe451e3a8 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:21.331093 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 686/1392] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:35:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:35:21.337608 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c06ef9-5149-4919-84c9-43931b2eca29 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:21.338455 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c06ef9-5149-4919-84c9-43931b2eca29 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:21.338630 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c06ef9-5149-4919-84c9-43931b2eca29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:21.360616 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-01c06ef9-5149-4919-84c9-43931b2eca29 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:35:21.362494 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-01c06ef9-5149-4919-84c9-43931b2eca29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:35:21.367459 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db4cfb-3ad1-44da-8432-563a542a0326 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:21.367753 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db4cfb-3ad1-44da-8432-563a542a0326 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:21.368014 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db4cfb-3ad1-44da-8432-563a542a0326 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:21.369487 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 687/1393] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:35:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 16:35:21.375991 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564858bb-5efd-4c88-8f0f-ae4486c04026 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:21.376445 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564858bb-5efd-4c88-8f0f-ae4486c04026 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:21.376733 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 707/1394] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:35:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:35:21.377392 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564858bb-5efd-4c88-8f0f-ae4486c04026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:21.398823 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-564858bb-5efd-4c88-8f0f-ae4486c04026 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:35:21.400144 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-564858bb-5efd-4c88-8f0f-ae4486c04026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:35:21.413281 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 688/1395] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:35:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:35:21.424849 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c06ef9-5149-4919-84c9-43931b2eca29 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:35:21.443585 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c06ef9-5149-4919-84c9-43931b2eca29 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:35:21.449919 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681f8c63-57b0-4dbb-a5d8-5929ad0c2288 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:35:21.450644 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f8c63-57b0-4dbb-a5d8-5929ad0c2288 service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:21.450985 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f8c63-57b0-4dbb-a5d8-5929ad0c2288 service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:21.451271 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f8c63-57b0-4dbb-a5d8-5929ad0c2288 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:21.464109 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681f8c63-57b0-4dbb-a5d8-5929ad0c2288 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:35:21.467322 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681f8c63-57b0-4dbb-a5d8-5929ad0c2288 service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:35:21.483987 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4009c0e-2d02-4cb2-8229-ce0b64eda0ef None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:21.484470 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4009c0e-2d02-4cb2-8229-ce0b64eda0ef None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:21.487694 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4009c0e-2d02-4cb2-8229-ce0b64eda0ef None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:21.488725 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 708/1396] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:35:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:35:21.496291 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 689/1397] 10.222.0.36 () {54 vars in 962 bytes} [Thu May 19 16:35:21 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:35:21.503987 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c9d9f-537b-4efe-81de-eee1975ad0d1 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:21.504282 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c9d9f-537b-4efe-81de-eee1975ad0d1 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:21.504596 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c9d9f-537b-4efe-81de-eee1975ad0d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:21.584708 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-858c9d9f-537b-4efe-81de-eee1975ad0d1 None None] MFA Rules not processed for user `57a5d7151478434f86be085bcc487f0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:35:21.586272 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-858c9d9f-537b-4efe-81de-eee1975ad0d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:35:21.679484 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 709/1398] 10.222.0.36 () {58 vars in 1059 bytes} [Thu May 19 16:35:21 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:35:21.691515 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4009c0e-2d02-4cb2-8229-ce0b64eda0ef None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:35:21.712920 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4009c0e-2d02-4cb2-8229-ce0b64eda0ef None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:35:21.719574 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a1685c-4641-4e80-850c-28334bb0466b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57a5d7151478434f86be085bcc487f0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:35:21.720840 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1685c-4641-4e80-850c-28334bb0466b service glance] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:21.721339 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1685c-4641-4e80-850c-28334bb0466b service glance] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:21.721917 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1685c-4641-4e80-850c-28334bb0466b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:21.738362 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a1685c-4641-4e80-850c-28334bb0466b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:35:21.740357 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a1685c-4641-4e80-850c-28334bb0466b service glance] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:35:21.752764 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 690/1399] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:35:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:35:22.501136 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbbc299-9ac1-4517-a96c-f12f450c3d4e None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:22.501812 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbbc299-9ac1-4517-a96c-f12f450c3d4e None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:22.501812 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbbc299-9ac1-4517-a96c-f12f450c3d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:22.561912 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-edbbc299-9ac1-4517-a96c-f12f450c3d4e None None] MFA Rules not processed for user `a30dc1dd2b00463bb3c7957864c56043`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:35:22.563261 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-edbbc299-9ac1-4517-a96c-f12f450c3d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:35:22.635167 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:35:22.641670 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 710/1400] 10.222.0.36 () {58 vars in 1009 bytes} [Thu May 19 16:35:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:35:22.695636 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c127a4b5-22b1-4cd2-857d-52bcf2c185d4 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:22.696402 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c127a4b5-22b1-4cd2-857d-52bcf2c185d4 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:22.696910 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c127a4b5-22b1-4cd2-857d-52bcf2c185d4 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:22.699670 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 691/1401] 10.222.0.36 () {54 vars in 942 bytes} [Thu May 19 16:35:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:35:22.709841 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8fa25d-8700-445a-9a2f-d00f1f55af59 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:22.710137 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8fa25d-8700-445a-9a2f-d00f1f55af59 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:22.710422 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8fa25d-8700-445a-9a2f-d00f1f55af59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:22.778805 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-2d8fa25d-8700-445a-9a2f-d00f1f55af59 None None] MFA Rules not processed for user `ffe7cebc6f3b40d8a8e1866ed2da9822`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:35:22.780596 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-2d8fa25d-8700-445a-9a2f-d00f1f55af59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:35:22.886807 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 711/1402] 10.222.0.36 () {58 vars in 1039 bytes} [Thu May 19 16:35:22 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:35:22.897025 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c127a4b5-22b1-4cd2-857d-52bcf2c185d4 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:35:22.904722 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=93 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:35:22.913750 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c127a4b5-22b1-4cd2-857d-52bcf2c185d4 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:35:22.928375 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39e58f6-819e-4c00-b444-548c88948d29 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffe7cebc6f3b40d8a8e1866ed2da9822', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:35:22.929058 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e58f6-819e-4c00-b444-548c88948d29 service swift] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:22.929381 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e58f6-819e-4c00-b444-548c88948d29 service swift] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:22.929655 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e58f6-819e-4c00-b444-548c88948d29 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:22.939212 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39e58f6-819e-4c00-b444-548c88948d29 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:35:22.940835 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39e58f6-819e-4c00-b444-548c88948d29 service swift] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:35:22.951197 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 692/1403] 10.222.0.36 () {60 vars in 1384 bytes} [Thu May 19 16:35:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 16:35:24.363359 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2976dcfc-5869-47e7-883a-115fd315181c None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:24.365269 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2976dcfc-5869-47e7-883a-115fd315181c None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:24.366575 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2976dcfc-5869-47e7-883a-115fd315181c None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:24.369409 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 712/1404] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:35:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:35:24.376189 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e304bf0-ede8-4d10-a3ed-239454266cf7 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:24.376755 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e304bf0-ede8-4d10-a3ed-239454266cf7 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:24.376755 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e304bf0-ede8-4d10-a3ed-239454266cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:24.400647 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-7e304bf0-ede8-4d10-a3ed-239454266cf7 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:35:24.402200 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-7e304bf0-ede8-4d10-a3ed-239454266cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:35:24.412467 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 693/1405] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:35:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:35:24.467089 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2976dcfc-5869-47e7-883a-115fd315181c None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:35:24.492435 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2976dcfc-5869-47e7-883a-115fd315181c None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:35:24.505040 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2c29b3-15f9-42b3-a5e2-81bb778a0117 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:35:24.506424 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c29b3-15f9-42b3-a5e2-81bb778a0117 service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:24.506977 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c29b3-15f9-42b3-a5e2-81bb778a0117 service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:24.507493 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c29b3-15f9-42b3-a5e2-81bb778a0117 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:24.524639 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2c29b3-15f9-42b3-a5e2-81bb778a0117 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:35:24.528184 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2c29b3-15f9-42b3-a5e2-81bb778a0117 service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:35:24.549268 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 713/1406] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:35:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:35:27.479158 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9a947-72fb-4a43-8a98-4253a9044993 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:27.480032 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9a947-72fb-4a43-8a98-4253a9044993 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:27.480032 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9a947-72fb-4a43-8a98-4253a9044993 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:27.485995 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 694/1407] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:35:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 16:35:27.487606 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea099019-e0ad-46bb-9246-b57171a7b9a6 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:27.488389 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea099019-e0ad-46bb-9246-b57171a7b9a6 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:27.488389 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea099019-e0ad-46bb-9246-b57171a7b9a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:27.513917 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-ea099019-e0ad-46bb-9246-b57171a7b9a6 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:35:27.515473 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-ea099019-e0ad-46bb-9246-b57171a7b9a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:35:27.524649 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 714/1408] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:35:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:35:27.574724 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a9a947-72fb-4a43-8a98-4253a9044993 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:35:27.596474 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a9a947-72fb-4a43-8a98-4253a9044993 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:35:27.602187 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7f082b-a28a-45fb-b42b-e5bb449b6b0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:35:27.602909 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f082b-a28a-45fb-b42b-e5bb449b6b0d service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:27.603196 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f082b-a28a-45fb-b42b-e5bb449b6b0d service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:27.603476 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f082b-a28a-45fb-b42b-e5bb449b6b0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:27.613515 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7f082b-a28a-45fb-b42b-e5bb449b6b0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:35:27.615149 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7f082b-a28a-45fb-b42b-e5bb449b6b0d service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:35:27.629233 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 695/1409] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:35:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 16:35:37.508707 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=54 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:35:37.621789 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=14 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:35:48.887277 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8606ab6a-e7a7-4e60-a1c9-dc5805387ba2 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:48.887870 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8606ab6a-e7a7-4e60-a1c9-dc5805387ba2 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:48.887870 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8606ab6a-e7a7-4e60-a1c9-dc5805387ba2 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:48.889589 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 715/1410] 10.222.0.36 () {54 vars in 918 bytes} [Thu May 19 16:35:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:35:48.896792 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48080c2e-c2cb-43f6-980f-d02471f98b51 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:48.897400 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48080c2e-c2cb-43f6-980f-d02471f98b51 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:48.897842 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48080c2e-c2cb-43f6-980f-d02471f98b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:48.930900 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-48080c2e-c2cb-43f6-980f-d02471f98b51 None None] MFA Rules not processed for user `c6e158e5c87f4954aeb297f42dabdc16`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:35:48.933043 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-48080c2e-c2cb-43f6-980f-d02471f98b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:35:48.949787 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 696/1411] 10.222.0.36 () {58 vars in 1015 bytes} [Thu May 19 16:35:48 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:35:48.968179 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dadc3bd-8f17-471f-9a4f-ef0991439111 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:48.968530 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dadc3bd-8f17-471f-9a4f-ef0991439111 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:48.968855 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dadc3bd-8f17-471f-9a4f-ef0991439111 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:48.970613 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 716/1412] 10.222.0.36 () {54 vars in 918 bytes} [Thu May 19 16:35:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:35:48.977208 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2739f1f6-36ba-4343-95e2-abfbcb92c72c None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:48.977611 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2739f1f6-36ba-4343-95e2-abfbcb92c72c None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:48.977986 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2739f1f6-36ba-4343-95e2-abfbcb92c72c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:48.999512 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-2739f1f6-36ba-4343-95e2-abfbcb92c72c None None] MFA Rules not processed for user `58bdeada00b74b7b8e186e6dc51e6dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:35:49.000785 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-2739f1f6-36ba-4343-95e2-abfbcb92c72c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:35:49.012316 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 697/1413] 10.222.0.36 () {58 vars in 1015 bytes} [Thu May 19 16:35:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:35:52.012718 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21adba-7f09-4e7e-99aa-2213b898a3a9 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:52.013897 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21adba-7f09-4e7e-99aa-2213b898a3a9 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:52.013897 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21adba-7f09-4e7e-99aa-2213b898a3a9 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:52.015491 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 717/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 16:35:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:35:52.022704 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7493573-23fb-4c42-8b41-5963253b2e17 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:52.023558 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7493573-23fb-4c42-8b41-5963253b2e17 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:52.023735 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7493573-23fb-4c42-8b41-5963253b2e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:52.044550 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-e7493573-23fb-4c42-8b41-5963253b2e17 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:35:52.045823 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-e7493573-23fb-4c42-8b41-5963253b2e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:35:52.054061 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 698/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 16:35:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:35:52.099667 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b21adba-7f09-4e7e-99aa-2213b898a3a9 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:35:52.116988 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b21adba-7f09-4e7e-99aa-2213b898a3a9 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:35:52.122696 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91491b4a-a8f2-4228-8047-84f450580522 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:35:52.123371 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91491b4a-a8f2-4228-8047-84f450580522 service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:52.123657 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91491b4a-a8f2-4228-8047-84f450580522 service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:52.123944 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91491b4a-a8f2-4228-8047-84f450580522 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:52.134583 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91491b4a-a8f2-4228-8047-84f450580522 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:35:52.136272 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91491b4a-a8f2-4228-8047-84f450580522 service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:35:52.149011 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 718/1416] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:35:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:35:53.290637 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4654bb2a-395a-4a86-a065-c801b96eef49 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:53.291197 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4654bb2a-395a-4a86-a065-c801b96eef49 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:53.291639 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4654bb2a-395a-4a86-a065-c801b96eef49 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:53.294598 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 699/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 16:35:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:35:53.301592 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15ca37-da39-42ac-b599-e51580a456a5 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:53.302252 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15ca37-da39-42ac-b599-e51580a456a5 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:53.302252 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15ca37-da39-42ac-b599-e51580a456a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:53.325615 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-4b15ca37-da39-42ac-b599-e51580a456a5 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:35:53.327429 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-4b15ca37-da39-42ac-b599-e51580a456a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:35:53.338794 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 719/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 16:35:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:35:53.522793 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4654bb2a-395a-4a86-a065-c801b96eef49 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:35:53.540296 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4654bb2a-395a-4a86-a065-c801b96eef49 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:35:53.547188 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee97f807-5054-427b-a629-f7579cf13c0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:35:53.548031 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee97f807-5054-427b-a629-f7579cf13c0b service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:53.548478 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee97f807-5054-427b-a629-f7579cf13c0b service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:53.548808 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee97f807-5054-427b-a629-f7579cf13c0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:53.559207 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee97f807-5054-427b-a629-f7579cf13c0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:35:53.560856 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee97f807-5054-427b-a629-f7579cf13c0b service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:35:53.572436 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 700/1419] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:35:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:35:54.602370 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59250bc6-dcfd-4c2a-90cc-1d72330b0926 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:54.603242 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59250bc6-dcfd-4c2a-90cc-1d72330b0926 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:54.603985 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59250bc6-dcfd-4c2a-90cc-1d72330b0926 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:54.607063 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 720/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 16:35:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:35:54.614433 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cdf087-2278-4177-97cf-bd76e5375932 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:54.615062 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cdf087-2278-4177-97cf-bd76e5375932 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:54.615062 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cdf087-2278-4177-97cf-bd76e5375932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:54.636308 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-83cdf087-2278-4177-97cf-bd76e5375932 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:35:54.637666 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-83cdf087-2278-4177-97cf-bd76e5375932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:35:54.647018 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 701/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 16:35:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:35:54.811494 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59250bc6-dcfd-4c2a-90cc-1d72330b0926 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:35:54.827860 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59250bc6-dcfd-4c2a-90cc-1d72330b0926 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:35:54.834431 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c919477-d03f-4694-8450-7cf6456de45e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:35:54.835109 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c919477-d03f-4694-8450-7cf6456de45e service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:54.835397 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c919477-d03f-4694-8450-7cf6456de45e service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:54.835679 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c919477-d03f-4694-8450-7cf6456de45e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:54.845522 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c919477-d03f-4694-8450-7cf6456de45e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:35:54.847914 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c919477-d03f-4694-8450-7cf6456de45e service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:35:54.858965 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 721/1422] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:35:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:35:56.107152 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a33f7-b8aa-418e-bfdc-74c0fef2aee1 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:56.107707 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a33f7-b8aa-418e-bfdc-74c0fef2aee1 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:56.107867 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a33f7-b8aa-418e-bfdc-74c0fef2aee1 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:56.109406 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 702/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 16:35:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:35:56.115264 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f7918-1b85-4c99-a2dd-7db45a501547 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:56.115891 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f7918-1b85-4c99-a2dd-7db45a501547 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:56.115891 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f7918-1b85-4c99-a2dd-7db45a501547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:56.135952 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-0a8f7918-1b85-4c99-a2dd-7db45a501547 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:35:56.138014 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-0a8f7918-1b85-4c99-a2dd-7db45a501547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:35:56.148581 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 722/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 16:35:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:35:56.306507 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55a33f7-b8aa-418e-bfdc-74c0fef2aee1 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:35:56.322297 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55a33f7-b8aa-418e-bfdc-74c0fef2aee1 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:35:56.328103 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca6eb4-43cf-4e44-b5dc-bf28caf126ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:35:56.328779 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca6eb4-43cf-4e44-b5dc-bf28caf126ed service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:56.329071 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca6eb4-43cf-4e44-b5dc-bf28caf126ed service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:56.329386 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca6eb4-43cf-4e44-b5dc-bf28caf126ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:56.338588 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ca6eb4-43cf-4e44-b5dc-bf28caf126ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:35:56.340225 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ca6eb4-43cf-4e44-b5dc-bf28caf126ed service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:35:56.350968 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 703/1425] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:35:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:35:57.395057 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb7c73-6a2e-499d-8656-0a9ce2ad0fd9 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:57.395958 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb7c73-6a2e-499d-8656-0a9ce2ad0fd9 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:57.395958 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb7c73-6a2e-499d-8656-0a9ce2ad0fd9 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:57.397808 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 723/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 16:35:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:35:57.404254 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df3517-f92c-4757-b418-03808fe07832 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:57.404689 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df3517-f92c-4757-b418-03808fe07832 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:57.405005 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df3517-f92c-4757-b418-03808fe07832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:57.426896 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-a2df3517-f92c-4757-b418-03808fe07832 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:35:57.428151 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-a2df3517-f92c-4757-b418-03808fe07832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:35:57.436199 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 704/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 16:35:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:35:57.588768 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb7c73-6a2e-499d-8656-0a9ce2ad0fd9 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:35:57.605633 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb7c73-6a2e-499d-8656-0a9ce2ad0fd9 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:35:57.614196 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ec9e48-3ce6-46a9-ab8d-861ac60bee53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:35:57.614878 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ec9e48-3ce6-46a9-ab8d-861ac60bee53 service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:57.615172 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ec9e48-3ce6-46a9-ab8d-861ac60bee53 service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:57.615453 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ec9e48-3ce6-46a9-ab8d-861ac60bee53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:57.627489 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ec9e48-3ce6-46a9-ab8d-861ac60bee53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:35:57.629253 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ec9e48-3ce6-46a9-ab8d-861ac60bee53 service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:35:57.640947 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 724/1428] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:35:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:35:58.540081 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa8688-2e92-4450-a9d1-ef6868d0b0d8 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:58.540644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa8688-2e92-4450-a9d1-ef6868d0b0d8 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:58.540644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa8688-2e92-4450-a9d1-ef6868d0b0d8 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:58.542092 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 705/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 16:35:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:35:58.546882 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc46b0-fc88-4feb-88ec-1c79ef43f18f None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:58.547283 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc46b0-fc88-4feb-88ec-1c79ef43f18f None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:58.547447 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc46b0-fc88-4feb-88ec-1c79ef43f18f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:58.565259 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-12fc46b0-fc88-4feb-88ec-1c79ef43f18f None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:35:58.566500 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-12fc46b0-fc88-4feb-88ec-1c79ef43f18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:35:58.574641 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 725/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 16:35:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:35:58.726592 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa8688-2e92-4450-a9d1-ef6868d0b0d8 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:35:58.740957 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa8688-2e92-4450-a9d1-ef6868d0b0d8 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:35:58.746295 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9859f3e7-e3ef-47f7-85d9-70e1ae231f87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:35:58.746932 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9859f3e7-e3ef-47f7-85d9-70e1ae231f87 service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:58.747221 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9859f3e7-e3ef-47f7-85d9-70e1ae231f87 service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:58.747505 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9859f3e7-e3ef-47f7-85d9-70e1ae231f87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:58.756068 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9859f3e7-e3ef-47f7-85d9-70e1ae231f87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:35:58.757697 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9859f3e7-e3ef-47f7-85d9-70e1ae231f87 service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:35:58.767405 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 706/1431] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:35:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:35:59.672847 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b1eb6-dd76-47be-97df-b7822f50b0d5 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:59.673471 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b1eb6-dd76-47be-97df-b7822f50b0d5 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:59.673471 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b1eb6-dd76-47be-97df-b7822f50b0d5 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:59.674972 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 726/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 16:35:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 16:35:59.681326 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4ff85-dee8-4b1f-ac4d-dc7570a2d417 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:59.682214 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4ff85-dee8-4b1f-ac4d-dc7570a2d417 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:59.682599 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4ff85-dee8-4b1f-ac4d-dc7570a2d417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:59.702587 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-eea4ff85-dee8-4b1f-ac4d-dc7570a2d417 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:35:59.703808 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-eea4ff85-dee8-4b1f-ac4d-dc7570a2d417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:35:59.712501 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 707/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 16:35:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:35:59.862549 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b1eb6-dd76-47be-97df-b7822f50b0d5 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:35:59.878770 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b1eb6-dd76-47be-97df-b7822f50b0d5 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:35:59.884630 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7356e69-3fa7-40bc-b7df-cc6cb57f2480 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:35:59.885341 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7356e69-3fa7-40bc-b7df-cc6cb57f2480 service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:35:59.885632 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7356e69-3fa7-40bc-b7df-cc6cb57f2480 service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:35:59.885910 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7356e69-3fa7-40bc-b7df-cc6cb57f2480 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:35:59.895890 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7356e69-3fa7-40bc-b7df-cc6cb57f2480 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:35:59.897563 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7356e69-3fa7-40bc-b7df-cc6cb57f2480 service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:35:59.907691 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 727/1434] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:35:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:36:00.779887 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a843f-2b68-49a4-bd0f-85e67ded818a None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:36:00.780458 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a843f-2b68-49a4-bd0f-85e67ded818a None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:36:00.780458 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a843f-2b68-49a4-bd0f-85e67ded818a None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:36:00.781914 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 708/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 16:36:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:36:00.787373 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418bea3-e753-4236-822c-46e621c4505d None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:36:00.787950 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418bea3-e753-4236-822c-46e621c4505d None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:36:00.787950 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418bea3-e753-4236-822c-46e621c4505d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:36:00.807973 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-1418bea3-e753-4236-822c-46e621c4505d None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:36:00.809250 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-1418bea3-e753-4236-822c-46e621c4505d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:36:00.818091 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 728/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 16:36:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:36:00.970735 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3a843f-2b68-49a4-bd0f-85e67ded818a None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:36:00.985242 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3a843f-2b68-49a4-bd0f-85e67ded818a None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:36:00.991305 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424ba8fb-df60-43fb-8693-023837326c24 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:36:00.991960 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ba8fb-df60-43fb-8693-023837326c24 service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:36:00.992250 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ba8fb-df60-43fb-8693-023837326c24 service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:36:00.992646 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ba8fb-df60-43fb-8693-023837326c24 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:36:01.001665 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424ba8fb-df60-43fb-8693-023837326c24 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:36:01.003862 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424ba8fb-df60-43fb-8693-023837326c24 service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:36:01.014354 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 709/1437] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:36:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:36:01.910018 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca98a6-c691-40f3-9300-a41caa062bce None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:36:01.910618 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca98a6-c691-40f3-9300-a41caa062bce None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:36:01.910618 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca98a6-c691-40f3-9300-a41caa062bce None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:36:01.912015 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 729/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 16:36:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:36:01.918201 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c904166-89b5-4212-86eb-bc23e7fc6a10 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:36:01.918987 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c904166-89b5-4212-86eb-bc23e7fc6a10 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:36:01.919154 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c904166-89b5-4212-86eb-bc23e7fc6a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:36:01.948321 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-4c904166-89b5-4212-86eb-bc23e7fc6a10 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:36:01.950425 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-4c904166-89b5-4212-86eb-bc23e7fc6a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:36:01.964641 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 710/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 16:36:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:36:02.116514 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ca98a6-c691-40f3-9300-a41caa062bce None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:36:02.131944 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ca98a6-c691-40f3-9300-a41caa062bce None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:36:02.137342 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730c6a11-8221-4990-b34c-4013a8c61b31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:36:02.137987 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c6a11-8221-4990-b34c-4013a8c61b31 service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:36:02.138533 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c6a11-8221-4990-b34c-4013a8c61b31 service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:36:02.138812 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c6a11-8221-4990-b34c-4013a8c61b31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:36:02.148192 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730c6a11-8221-4990-b34c-4013a8c61b31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:36:02.149974 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730c6a11-8221-4990-b34c-4013a8c61b31 service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:36:02.159951 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 730/1440] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:36:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:36:04.360260 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ecb1a5-3227-484f-9740-8839871c7d7c None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:36:04.360779 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ecb1a5-3227-484f-9740-8839871c7d7c None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:36:04.361002 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ecb1a5-3227-484f-9740-8839871c7d7c None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:36:04.363697 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 711/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 16:36:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:36:04.369217 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555522e-87eb-4f24-bc2f-c0a7e8d64ec7 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:36:04.369641 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555522e-87eb-4f24-bc2f-c0a7e8d64ec7 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:36:04.369823 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555522e-87eb-4f24-bc2f-c0a7e8d64ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:36:04.388063 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-b555522e-87eb-4f24-bc2f-c0a7e8d64ec7 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:36:04.389573 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b555522e-87eb-4f24-bc2f-c0a7e8d64ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:36:04.397911 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 731/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 16:36:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:36:04.435952 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ecb1a5-3227-484f-9740-8839871c7d7c None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:36:04.447906 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ecb1a5-3227-484f-9740-8839871c7d7c None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:36:04.451702 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf1a681-2aff-4b76-815c-b1408eb33471 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:36:04.452387 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1a681-2aff-4b76-815c-b1408eb33471 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:36:04.452677 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1a681-2aff-4b76-815c-b1408eb33471 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:36:04.452960 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1a681-2aff-4b76-815c-b1408eb33471 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:36:04.464136 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf1a681-2aff-4b76-815c-b1408eb33471 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:36:04.465909 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf1a681-2aff-4b76-815c-b1408eb33471 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:36:04.478776 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 712/1443] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:36:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:36:06.888717 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571841e3-458f-459f-b7ea-41e10e106932 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:36:06.889349 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571841e3-458f-459f-b7ea-41e10e106932 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:36:06.889349 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571841e3-458f-459f-b7ea-41e10e106932 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:36:06.890734 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 732/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 16:36:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:36:06.896855 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca559d1-f847-4369-8de3-bd8c10621751 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:36:06.897346 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca559d1-f847-4369-8de3-bd8c10621751 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:36:06.897461 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca559d1-f847-4369-8de3-bd8c10621751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:36:06.915587 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-2ca559d1-f847-4369-8de3-bd8c10621751 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:36:06.916836 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-2ca559d1-f847-4369-8de3-bd8c10621751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:36:06.924810 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 713/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 16:36:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:36:06.963589 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571841e3-458f-459f-b7ea-41e10e106932 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:36:06.975337 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571841e3-458f-459f-b7ea-41e10e106932 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:36:06.978907 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11148bd6-6246-4116-a2e6-b82cef13e43c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:36:06.979543 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11148bd6-6246-4116-a2e6-b82cef13e43c service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:36:06.979833 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11148bd6-6246-4116-a2e6-b82cef13e43c service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:36:06.980114 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11148bd6-6246-4116-a2e6-b82cef13e43c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:36:06.989109 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11148bd6-6246-4116-a2e6-b82cef13e43c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:36:06.990606 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11148bd6-6246-4116-a2e6-b82cef13e43c service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:36:07.000322 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 733/1446] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:36:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:36:09.668161 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0523ae-b05a-4380-955d-f9127ded7092 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:36:09.669325 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0523ae-b05a-4380-955d-f9127ded7092 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:36:09.669325 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0523ae-b05a-4380-955d-f9127ded7092 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:36:09.671045 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 714/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 16:36:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:36:09.676568 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78310eb6-a02e-458f-a99e-12da0ae2e979 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:36:09.677008 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78310eb6-a02e-458f-a99e-12da0ae2e979 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:36:09.677122 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78310eb6-a02e-458f-a99e-12da0ae2e979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:36:09.694695 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-78310eb6-a02e-458f-a99e-12da0ae2e979 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:36:09.695895 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-78310eb6-a02e-458f-a99e-12da0ae2e979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:36:09.703670 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 734/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 16:36:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:36:09.742492 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0523ae-b05a-4380-955d-f9127ded7092 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:36:09.753557 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0523ae-b05a-4380-955d-f9127ded7092 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:36:09.757459 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb7b7cd-d657-4abe-9f98-156ca3d7d46c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:36:09.758152 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7b7cd-d657-4abe-9f98-156ca3d7d46c service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:36:09.758435 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7b7cd-d657-4abe-9f98-156ca3d7d46c service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:36:09.758778 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7b7cd-d657-4abe-9f98-156ca3d7d46c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:36:09.770413 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb7b7cd-d657-4abe-9f98-156ca3d7d46c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:36:09.771866 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb7b7cd-d657-4abe-9f98-156ca3d7d46c service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:36:09.781849 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 715/1449] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:36:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:36:12.210095 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da6d52-c559-480d-b9f4-16c784dcc4d4 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:36:12.210628 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da6d52-c559-480d-b9f4-16c784dcc4d4 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:36:12.210628 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da6d52-c559-480d-b9f4-16c784dcc4d4 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:36:12.211958 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 735/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 16:36:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:36:12.217456 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d38ec44-0ceb-4958-bb51-a4df1538c662 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:36:12.217882 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d38ec44-0ceb-4958-bb51-a4df1538c662 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:36:12.218023 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d38ec44-0ceb-4958-bb51-a4df1538c662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:36:12.236302 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-5d38ec44-0ceb-4958-bb51-a4df1538c662 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:36:12.237574 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-5d38ec44-0ceb-4958-bb51-a4df1538c662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:36:12.245367 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 716/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 16:36:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:36:12.284560 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da6d52-c559-480d-b9f4-16c784dcc4d4 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:36:12.295255 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da6d52-c559-480d-b9f4-16c784dcc4d4 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:36:12.298965 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f528347-938a-482d-8cf6-61f520553f6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:36:12.299646 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f528347-938a-482d-8cf6-61f520553f6f service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:36:12.299943 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f528347-938a-482d-8cf6-61f520553f6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:36:12.300231 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f528347-938a-482d-8cf6-61f520553f6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:36:12.309228 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f528347-938a-482d-8cf6-61f520553f6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:36:12.310686 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f528347-938a-482d-8cf6-61f520553f6f service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:36:12.320905 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 736/1452] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:36:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:36:14.753514 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78136cf0-676e-4563-a590-307adbd55b2e None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:36:14.754099 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78136cf0-676e-4563-a590-307adbd55b2e None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:36:14.754099 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78136cf0-676e-4563-a590-307adbd55b2e None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:36:14.755479 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 717/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 16:36:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:36:14.761091 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f7624-a909-4bbe-acdf-087ca4f9c631 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:36:14.761533 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f7624-a909-4bbe-acdf-087ca4f9c631 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:36:14.761685 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f7624-a909-4bbe-acdf-087ca4f9c631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:36:14.780932 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-8d6f7624-a909-4bbe-acdf-087ca4f9c631 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:36:14.782185 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-8d6f7624-a909-4bbe-acdf-087ca4f9c631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:36:14.792020 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 737/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 16:36:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:36:14.835155 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78136cf0-676e-4563-a590-307adbd55b2e None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:36:14.846032 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78136cf0-676e-4563-a590-307adbd55b2e None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:36:14.854180 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c5058-d21c-491f-93c7-8a55f9694d80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:36:14.854859 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c5058-d21c-491f-93c7-8a55f9694d80 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:36:14.855144 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c5058-d21c-491f-93c7-8a55f9694d80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:36:14.855451 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c5058-d21c-491f-93c7-8a55f9694d80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:36:14.865137 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8c5058-d21c-491f-93c7-8a55f9694d80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:36:14.866601 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8c5058-d21c-491f-93c7-8a55f9694d80 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:36:14.875870 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 718/1455] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:36:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:36:17.314812 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2786ca9c-f7bf-49fe-a37d-d6f2eb3f644b None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:36:17.315390 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2786ca9c-f7bf-49fe-a37d-d6f2eb3f644b None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:36:17.315390 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2786ca9c-f7bf-49fe-a37d-d6f2eb3f644b None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:36:17.316795 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 738/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 16:36:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:36:17.322860 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff4e53-b6c7-4599-8347-2895de46f30b None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:36:17.323361 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff4e53-b6c7-4599-8347-2895de46f30b None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:36:17.323470 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff4e53-b6c7-4599-8347-2895de46f30b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:36:17.344004 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-7eff4e53-b6c7-4599-8347-2895de46f30b None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:36:17.345252 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-7eff4e53-b6c7-4599-8347-2895de46f30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:36:17.353913 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 719/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 16:36:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:36:17.390646 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2786ca9c-f7bf-49fe-a37d-d6f2eb3f644b None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:36:17.401678 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2786ca9c-f7bf-49fe-a37d-d6f2eb3f644b None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:36:17.405260 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe342da-803d-4d86-91f6-d2b3bb8da1cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:36:17.405918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe342da-803d-4d86-91f6-d2b3bb8da1cf service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:36:17.406207 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe342da-803d-4d86-91f6-d2b3bb8da1cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:36:17.406487 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe342da-803d-4d86-91f6-d2b3bb8da1cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:36:17.416423 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe342da-803d-4d86-91f6-d2b3bb8da1cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:36:17.417922 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe342da-803d-4d86-91f6-d2b3bb8da1cf service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:36:17.427359 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 739/1458] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:36:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:36:19.721222 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3cc75-b7e5-4006-88b5-e65f946cb594 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:36:19.722334 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3cc75-b7e5-4006-88b5-e65f946cb594 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:36:19.722334 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3cc75-b7e5-4006-88b5-e65f946cb594 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:36:19.723707 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 720/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 16:36:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:36:19.728911 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d2842-588a-4016-9dc9-b86c64874707 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:36:19.729329 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d2842-588a-4016-9dc9-b86c64874707 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:36:19.729519 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d2842-588a-4016-9dc9-b86c64874707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:36:19.735546 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=70 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:36:19.748018 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-f93d2842-588a-4016-9dc9-b86c64874707 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:36:19.749372 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-f93d2842-588a-4016-9dc9-b86c64874707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:36:19.757803 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 740/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 16:36:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:36:19.794868 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e3cc75-b7e5-4006-88b5-e65f946cb594 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:36:19.800237 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=73 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:36:19.808038 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e3cc75-b7e5-4006-88b5-e65f946cb594 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:36:19.813469 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a388d348-eacd-45d2-823a-876fd3f4efc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:36:19.814126 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388d348-eacd-45d2-823a-876fd3f4efc4 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:36:19.814413 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388d348-eacd-45d2-823a-876fd3f4efc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:36:19.814692 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388d348-eacd-45d2-823a-876fd3f4efc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:36:19.827206 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a388d348-eacd-45d2-823a-876fd3f4efc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:36:19.828654 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a388d348-eacd-45d2-823a-876fd3f4efc4 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:36:19.839490 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 721/1461] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:36:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:36:22.125783 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1639b-038f-47df-8207-b06ecc1d792f None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:36:22.126286 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1639b-038f-47df-8207-b06ecc1d792f None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:36:22.126385 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1639b-038f-47df-8207-b06ecc1d792f None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:36:22.127699 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 741/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 16:36:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:36:22.133045 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77080b-212f-4d0c-8024-56967c2dbbcd None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:36:22.133465 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77080b-212f-4d0c-8024-56967c2dbbcd None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:36:22.133643 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77080b-212f-4d0c-8024-56967c2dbbcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:36:22.151361 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-ec77080b-212f-4d0c-8024-56967c2dbbcd None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:36:22.152482 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ec77080b-212f-4d0c-8024-56967c2dbbcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:36:22.160745 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 722/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 16:36:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:36:22.197816 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f1639b-038f-47df-8207-b06ecc1d792f None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:36:22.211642 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f1639b-038f-47df-8207-b06ecc1d792f None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:36:22.215328 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c3634e-f020-44d6-a91b-572023e2dea1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:36:22.215972 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3634e-f020-44d6-a91b-572023e2dea1 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:36:22.216284 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3634e-f020-44d6-a91b-572023e2dea1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:36:22.216563 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3634e-f020-44d6-a91b-572023e2dea1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:36:22.227773 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c3634e-f020-44d6-a91b-572023e2dea1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:36:22.229277 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c3634e-f020-44d6-a91b-572023e2dea1 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:36:22.239873 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 742/1464] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:36:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:36:24.569226 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f4e01-9867-4f66-8a59-71a2d0fbeb1d None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:36:24.569817 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f4e01-9867-4f66-8a59-71a2d0fbeb1d None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:36:24.569817 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f4e01-9867-4f66-8a59-71a2d0fbeb1d None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:36:24.571156 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 723/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 16:36:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:36:24.576676 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2a37d-2c6c-48e7-bbc8-1f94b9c00924 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:36:24.577130 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2a37d-2c6c-48e7-bbc8-1f94b9c00924 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:36:24.577257 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2a37d-2c6c-48e7-bbc8-1f94b9c00924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:36:24.595692 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-24c2a37d-2c6c-48e7-bbc8-1f94b9c00924 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:36:24.597492 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-24c2a37d-2c6c-48e7-bbc8-1f94b9c00924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:36:24.605504 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 743/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 16:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:36:24.644231 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3f4e01-9867-4f66-8a59-71a2d0fbeb1d None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:36:24.654908 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3f4e01-9867-4f66-8a59-71a2d0fbeb1d None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:36:24.658424 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b798bb7b-cb5c-4f6a-be18-45c60014ab41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:36:24.659629 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798bb7b-cb5c-4f6a-be18-45c60014ab41 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:36:24.659910 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798bb7b-cb5c-4f6a-be18-45c60014ab41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:36:24.660185 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798bb7b-cb5c-4f6a-be18-45c60014ab41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:36:24.669131 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b798bb7b-cb5c-4f6a-be18-45c60014ab41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:36:24.670651 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b798bb7b-cb5c-4f6a-be18-45c60014ab41 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:36:24.680094 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 724/1467] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:36:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:36:26.973542 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fef429-7547-4e3d-8d55-81d03667abf2 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:36:26.974111 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fef429-7547-4e3d-8d55-81d03667abf2 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:36:26.974111 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fef429-7547-4e3d-8d55-81d03667abf2 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:36:26.975501 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 744/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 16:36:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:36:26.980867 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06d554-3d8c-49d9-b4c4-f6fbdff17cd6 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:36:26.981328 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06d554-3d8c-49d9-b4c4-f6fbdff17cd6 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:36:26.981489 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06d554-3d8c-49d9-b4c4-f6fbdff17cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:36:26.999453 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-9b06d554-3d8c-49d9-b4c4-f6fbdff17cd6 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:36:27.000658 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-9b06d554-3d8c-49d9-b4c4-f6fbdff17cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:36:27.008968 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 725/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 16:36:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:36:27.044967 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fef429-7547-4e3d-8d55-81d03667abf2 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:36:27.056652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fef429-7547-4e3d-8d55-81d03667abf2 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:36:27.060480 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb03904-76cc-4b44-9c84-dc1a93a247b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:36:27.061120 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb03904-76cc-4b44-9c84-dc1a93a247b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:36:27.061530 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb03904-76cc-4b44-9c84-dc1a93a247b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:36:27.061724 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb03904-76cc-4b44-9c84-dc1a93a247b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:36:27.072426 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb03904-76cc-4b44-9c84-dc1a93a247b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:36:27.074033 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb03904-76cc-4b44-9c84-dc1a93a247b2 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:36:27.084450 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 745/1470] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:36:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:36:36.994268 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=17 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:36:37.079750 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=17 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:37:01.383186 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c96bfd-48c7-46e3-a77a-22abf2d4d39e None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:37:01.383759 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c96bfd-48c7-46e3-a77a-22abf2d4d39e None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:37:01.383759 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c96bfd-48c7-46e3-a77a-22abf2d4d39e None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:37:01.385121 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 726/1471] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:37:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:37:01.390784 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18acb57-3680-45d1-91cc-5d6945c9d80e None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:37:01.391209 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18acb57-3680-45d1-91cc-5d6945c9d80e None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:37:01.391209 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18acb57-3680-45d1-91cc-5d6945c9d80e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:37:01.410135 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-d18acb57-3680-45d1-91cc-5d6945c9d80e None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:37:01.411373 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d18acb57-3680-45d1-91cc-5d6945c9d80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:37:01.419342 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 746/1472] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:37:01.467507 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d21410b-1bda-4682-b587-886ab9bfce8e None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:37:01.467800 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d21410b-1bda-4682-b587-886ab9bfce8e None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:37:01.468081 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d21410b-1bda-4682-b587-886ab9bfce8e None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:37:01.469664 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 727/1473] 10.222.0.36 () {54 vars in 962 bytes} [Thu May 19 16:37:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:37:01.474454 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ba368a-5ac3-4336-9ea2-dfb494b17795 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:37:01.474899 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ba368a-5ac3-4336-9ea2-dfb494b17795 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:37:01.475192 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ba368a-5ac3-4336-9ea2-dfb494b17795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:37:01.495957 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-18ba368a-5ac3-4336-9ea2-dfb494b17795 None None] MFA Rules not processed for user `57a5d7151478434f86be085bcc487f0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:37:01.497192 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-18ba368a-5ac3-4336-9ea2-dfb494b17795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:37:01.504380 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 747/1474] 10.222.0.36 () {58 vars in 1059 bytes} [Thu May 19 16:37:01 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:37:01.513551 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d21410b-1bda-4682-b587-886ab9bfce8e None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:37:01.524337 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d21410b-1bda-4682-b587-886ab9bfce8e None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:37:01.527935 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ac575c-a90c-4d51-9d05-f9cc1603bbca service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57a5d7151478434f86be085bcc487f0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:37:01.528536 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ac575c-a90c-4d51-9d05-f9cc1603bbca service glance] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:37:01.528823 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ac575c-a90c-4d51-9d05-f9cc1603bbca service glance] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:37:01.529102 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ac575c-a90c-4d51-9d05-f9cc1603bbca service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:37:01.538745 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ac575c-a90c-4d51-9d05-f9cc1603bbca service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:37:01.540221 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ac575c-a90c-4d51-9d05-f9cc1603bbca service glance] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:37:01.550796 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 728/1475] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:37:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:37:03.948125 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80369c-2987-4506-a080-7396bddd651c None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:37:03.948699 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80369c-2987-4506-a080-7396bddd651c None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:37:03.948699 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80369c-2987-4506-a080-7396bddd651c None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:37:03.950172 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 748/1476] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:37:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:37:03.956032 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a6d92-b1f0-4747-abe8-9e537418e0d8 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:37:03.956538 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a6d92-b1f0-4747-abe8-9e537418e0d8 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:37:03.956659 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a6d92-b1f0-4747-abe8-9e537418e0d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:37:03.975515 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-378a6d92-b1f0-4747-abe8-9e537418e0d8 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:37:03.976727 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-378a6d92-b1f0-4747-abe8-9e537418e0d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:37:03.984485 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 729/1477] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:37:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:37:04.008644 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab80369c-2987-4506-a080-7396bddd651c None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:37:04.021724 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab80369c-2987-4506-a080-7396bddd651c None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:37:04.026254 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c8cb71-5f25-41e8-b842-7c864d87678c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:37:04.026912 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8cb71-5f25-41e8-b842-7c864d87678c admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:37:04.027196 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8cb71-5f25-41e8-b842-7c864d87678c admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:37:04.027476 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8cb71-5f25-41e8-b842-7c864d87678c admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:37:04.028229 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c8cb71-5f25-41e8-b842-7c864d87678c admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:37:04.030083 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: WARNING py.warnings [None req-d8c8cb71-5f25-41e8-b842-7c864d87678c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 16:37:04.030083 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: warnings.warn(msg) May 19 16:37:04.030083 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: May 19 16:37:04.031248 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c8cb71-5f25-41e8-b842-7c864d87678c admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:37:04.052909 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 749/1478] 10.222.0.36 () {56 vars in 1108 bytes} [Thu May 19 16:37:04 2022] GET /identity/v3/projects => generated 2553 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:37:06.358826 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32513e8-9691-4d6b-8624-f479666e4280 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:37:06.359352 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32513e8-9691-4d6b-8624-f479666e4280 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:37:06.359352 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32513e8-9691-4d6b-8624-f479666e4280 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:37:06.360658 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 730/1479] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:37:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:37:06.366038 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedadd96-de41-4870-9ed7-5be8fbc8cd2f None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:37:06.366446 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedadd96-de41-4870-9ed7-5be8fbc8cd2f None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:37:06.366611 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedadd96-de41-4870-9ed7-5be8fbc8cd2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:37:06.384828 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-bedadd96-de41-4870-9ed7-5be8fbc8cd2f None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:37:06.386057 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-bedadd96-de41-4870-9ed7-5be8fbc8cd2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:37:06.394679 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 750/1480] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:37:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:37:06.437106 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32513e8-9691-4d6b-8624-f479666e4280 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:37:06.458840 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32513e8-9691-4d6b-8624-f479666e4280 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:37:06.464804 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd29fc9-3f3d-4b15-8861-dbde2355689c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:37:06.465612 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd29fc9-3f3d-4b15-8861-dbde2355689c service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:37:06.465954 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd29fc9-3f3d-4b15-8861-dbde2355689c service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:37:06.466273 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd29fc9-3f3d-4b15-8861-dbde2355689c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:37:06.482762 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd29fc9-3f3d-4b15-8861-dbde2355689c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:37:06.484365 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd29fc9-3f3d-4b15-8861-dbde2355689c service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:37:06.496236 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 731/1481] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:37:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:37:08.957110 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b0257-c3c3-43ef-bea9-a3605de73064 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:37:08.957733 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b0257-c3c3-43ef-bea9-a3605de73064 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:37:08.957733 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b0257-c3c3-43ef-bea9-a3605de73064 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:37:08.959088 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 751/1482] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:37:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:37:08.964634 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d5a97-5a3a-4bd1-8b1b-0f9a82d18c6d None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:37:08.965109 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d5a97-5a3a-4bd1-8b1b-0f9a82d18c6d None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:37:08.965327 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d5a97-5a3a-4bd1-8b1b-0f9a82d18c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:37:08.983537 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-dd1d5a97-5a3a-4bd1-8b1b-0f9a82d18c6d None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:37:08.984768 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-dd1d5a97-5a3a-4bd1-8b1b-0f9a82d18c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:37:08.992581 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 732/1483] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:37:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:37:09.028879 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6b0257-c3c3-43ef-bea9-a3605de73064 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:37:09.040355 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6b0257-c3c3-43ef-bea9-a3605de73064 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:37:09.043935 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56560af9-52de-43c2-9c88-38b9b87d0a09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:37:09.044581 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56560af9-52de-43c2-9c88-38b9b87d0a09 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:37:09.044873 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56560af9-52de-43c2-9c88-38b9b87d0a09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:37:09.045181 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56560af9-52de-43c2-9c88-38b9b87d0a09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:37:09.054852 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56560af9-52de-43c2-9c88-38b9b87d0a09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:37:09.056323 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56560af9-52de-43c2-9c88-38b9b87d0a09 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:37:09.066495 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 752/1484] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:37:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 16:37:11.378177 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e38618a-a30d-49d9-a986-9be9667cea42 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:37:11.378842 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e38618a-a30d-49d9-a986-9be9667cea42 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:37:11.378842 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e38618a-a30d-49d9-a986-9be9667cea42 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:37:11.380116 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 733/1485] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:37:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:37:11.385251 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0cddc-bf32-4916-80c8-7074f14c52eb None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:37:11.385705 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0cddc-bf32-4916-80c8-7074f14c52eb None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:37:11.385822 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0cddc-bf32-4916-80c8-7074f14c52eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:37:11.403738 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-39c0cddc-bf32-4916-80c8-7074f14c52eb None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:37:11.404972 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-39c0cddc-bf32-4916-80c8-7074f14c52eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:37:11.414516 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 753/1486] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:37:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:37:11.452937 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e38618a-a30d-49d9-a986-9be9667cea42 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:37:11.463629 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e38618a-a30d-49d9-a986-9be9667cea42 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:37:11.467271 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca6c651-b1d1-46b6-97da-b66420d4a646 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:37:11.467911 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6c651-b1d1-46b6-97da-b66420d4a646 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:37:11.468196 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6c651-b1d1-46b6-97da-b66420d4a646 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:37:11.468476 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6c651-b1d1-46b6-97da-b66420d4a646 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:37:11.477383 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca6c651-b1d1-46b6-97da-b66420d4a646 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:37:11.478866 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca6c651-b1d1-46b6-97da-b66420d4a646 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:37:11.490925 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 734/1487] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:37:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:37:13.912699 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff805a-bb3e-4f1d-80e7-e40ce21ef2eb None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:37:13.913290 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff805a-bb3e-4f1d-80e7-e40ce21ef2eb None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:37:13.913290 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff805a-bb3e-4f1d-80e7-e40ce21ef2eb None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:37:13.914701 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 754/1488] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:37:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:37:13.921047 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a258d-fe0c-4f41-ac22-c1db1feb4b27 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:37:13.921773 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a258d-fe0c-4f41-ac22-c1db1feb4b27 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:37:13.921773 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a258d-fe0c-4f41-ac22-c1db1feb4b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:37:13.941834 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-702a258d-fe0c-4f41-ac22-c1db1feb4b27 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:37:13.943108 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-702a258d-fe0c-4f41-ac22-c1db1feb4b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:37:13.955871 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 735/1489] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:37:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:37:13.997882 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff805a-bb3e-4f1d-80e7-e40ce21ef2eb None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:37:14.009437 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff805a-bb3e-4f1d-80e7-e40ce21ef2eb None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:37:14.013660 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf5d5f-f2bc-44b9-86c1-a9965d205a34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:37:14.014326 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf5d5f-f2bc-44b9-86c1-a9965d205a34 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:37:14.014619 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf5d5f-f2bc-44b9-86c1-a9965d205a34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:37:14.014906 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf5d5f-f2bc-44b9-86c1-a9965d205a34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:37:14.025020 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf5d5f-f2bc-44b9-86c1-a9965d205a34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:37:14.026546 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf5d5f-f2bc-44b9-86c1-a9965d205a34 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:37:14.037599 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 755/1490] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:37:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 16:37:16.554087 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cebe5da-f1e0-476b-914a-598493c083ff None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:37:16.554633 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cebe5da-f1e0-476b-914a-598493c083ff None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:37:16.554633 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cebe5da-f1e0-476b-914a-598493c083ff None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:37:16.556037 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 736/1491] 10.222.0.36 () {54 vars in 926 bytes} [Thu May 19 16:37:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:37:16.561516 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0cee5c-7a83-476d-ac5c-3bf19874afd1 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:37:16.562003 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0cee5c-7a83-476d-ac5c-3bf19874afd1 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:37:16.562110 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0cee5c-7a83-476d-ac5c-3bf19874afd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:37:16.583979 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-ff0cee5c-7a83-476d-ac5c-3bf19874afd1 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:37:16.585207 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-ff0cee5c-7a83-476d-ac5c-3bf19874afd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:37:16.592977 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 756/1492] 10.222.0.36 () {58 vars in 1023 bytes} [Thu May 19 16:37:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:37:16.630812 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cebe5da-f1e0-476b-914a-598493c083ff None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:37:16.642574 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cebe5da-f1e0-476b-914a-598493c083ff None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:37:16.646213 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239400cb-affb-493d-af0f-03e0116793ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:37:16.646856 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239400cb-affb-493d-af0f-03e0116793ed service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:37:16.647143 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239400cb-affb-493d-af0f-03e0116793ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:37:16.647424 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239400cb-affb-493d-af0f-03e0116793ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:37:16.656339 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239400cb-affb-493d-af0f-03e0116793ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:37:16.658404 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239400cb-affb-493d-af0f-03e0116793ed service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:37:16.667760 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 737/1493] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:37:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 16:37:26.577728 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=33 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:37:26.663607 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=33 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:39:16.657995 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c89e6-a6ee-4f46-b107-80abdb1449a8 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:39:16.658667 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c89e6-a6ee-4f46-b107-80abdb1449a8 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:39:16.658667 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c89e6-a6ee-4f46-b107-80abdb1449a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:39:16.677058 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-433c89e6-a6ee-4f46-b107-80abdb1449a8 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:39:16.678295 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-433c89e6-a6ee-4f46-b107-80abdb1449a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:39:16.687290 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 757/1494] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:39:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:39:16.697047 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7344830-8924-4184-864f-a11301896bea None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:39:16.697601 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7344830-8924-4184-864f-a11301896bea None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:39:16.697717 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7344830-8924-4184-864f-a11301896bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:39:16.715355 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-c7344830-8924-4184-864f-a11301896bea None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:39:16.716555 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-c7344830-8924-4184-864f-a11301896bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:39:16.725253 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 738/1495] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:39:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:39:16.729971 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433c89e6-a6ee-4f46-b107-80abdb1449a8 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:39:16.740236 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433c89e6-a6ee-4f46-b107-80abdb1449a8 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:39:16.744850 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e94a01-76ce-4ffc-a3e9-1a184ba58185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:39:16.745580 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e94a01-76ce-4ffc-a3e9-1a184ba58185 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:39:16.745846 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e94a01-76ce-4ffc-a3e9-1a184ba58185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:39:16.746123 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e94a01-76ce-4ffc-a3e9-1a184ba58185 admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:39:16.746854 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e94a01-76ce-4ffc-a3e9-1a184ba58185 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:39:16.747165 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e94a01-76ce-4ffc-a3e9-1a184ba58185 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:39:16.748849 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: WARNING py.warnings [None req-f6e94a01-76ce-4ffc-a3e9-1a184ba58185 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 19 16:39:16.748849 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: warnings.warn(msg) May 19 16:39:16.748849 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: May 19 16:39:16.749780 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e94a01-76ce-4ffc-a3e9-1a184ba58185 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:39:16.761904 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 758/1496] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:39:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:39:16.769952 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7344830-8924-4184-864f-a11301896bea None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:39:16.778943 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7344830-8924-4184-864f-a11301896bea None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:39:16.783993 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e170d-3575-48da-9d55-11c57572811d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:39:16.784641 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e170d-3575-48da-9d55-11c57572811d admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:39:16.784926 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e170d-3575-48da-9d55-11c57572811d admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:39:16.785233 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e170d-3575-48da-9d55-11c57572811d admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:39:16.785932 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374e170d-3575-48da-9d55-11c57572811d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:39:16.787699 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: WARNING py.warnings [None req-374e170d-3575-48da-9d55-11c57572811d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 16:39:16.787699 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: warnings.warn(msg) May 19 16:39:16.787699 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: May 19 16:39:16.788896 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374e170d-3575-48da-9d55-11c57572811d admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:39:16.823225 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 739/1497] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:39:16 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:39:16.827609 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e94a01-76ce-4ffc-a3e9-1a184ba58185 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:39:16.837664 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e94a01-76ce-4ffc-a3e9-1a184ba58185 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:39:16.897012 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a8ba8-d9bd-4259-8165-8ab650c961e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:39:16.897738 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a8ba8-d9bd-4259-8165-8ab650c961e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:39:16.898033 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a8ba8-d9bd-4259-8165-8ab650c961e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:39:16.898321 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a8ba8-d9bd-4259-8165-8ab650c961e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:39:16.899040 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1a8ba8-d9bd-4259-8165-8ab650c961e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:39:16.900093 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: WARNING py.warnings [None req-fb1a8ba8-d9bd-4259-8165-8ab650c961e9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 16:39:16.900093 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: warnings.warn(msg) May 19 16:39:16.900093 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: May 19 16:39:16.901444 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1a8ba8-d9bd-4259-8165-8ab650c961e9 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:39:16.922543 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 759/1498] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:39:16 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:39:16.926671 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e170d-3575-48da-9d55-11c57572811d admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:39:16.937118 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e170d-3575-48da-9d55-11c57572811d admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:39:16.941632 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9973780-22de-4169-89f9-fe0578aa0db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:39:16.942293 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9973780-22de-4169-89f9-fe0578aa0db2 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:39:16.942583 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9973780-22de-4169-89f9-fe0578aa0db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:39:16.942861 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9973780-22de-4169-89f9-fe0578aa0db2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:39:16.943584 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9973780-22de-4169-89f9-fe0578aa0db2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:39:16.944586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: WARNING py.warnings [None req-f9973780-22de-4169-89f9-fe0578aa0db2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 19 16:39:16.944586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: warnings.warn(msg) May 19 16:39:16.944586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: May 19 16:39:16.945506 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9973780-22de-4169-89f9-fe0578aa0db2 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:39:16.958065 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 740/1499] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:39:16.962075 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a8ba8-d9bd-4259-8165-8ab650c961e9 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:39:16.972561 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a8ba8-d9bd-4259-8165-8ab650c961e9 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:39:16.977522 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4fca70-eb09-4468-b1cd-35fc90cf1b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:39:16.978187 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fca70-eb09-4468-b1cd-35fc90cf1b73 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:39:16.978475 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fca70-eb09-4468-b1cd-35fc90cf1b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:39:16.978774 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fca70-eb09-4468-b1cd-35fc90cf1b73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:39:16.979497 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4fca70-eb09-4468-b1cd-35fc90cf1b73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:39:16.980518 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: WARNING py.warnings [None req-ea4fca70-eb09-4468-b1cd-35fc90cf1b73 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 19 16:39:16.980518 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: warnings.warn(msg) May 19 16:39:16.980518 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: May 19 16:39:16.981462 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4fca70-eb09-4468-b1cd-35fc90cf1b73 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:39:16.995624 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 760/1500] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:39:16.999573 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9973780-22de-4169-89f9-fe0578aa0db2 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:39:17.009455 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9973780-22de-4169-89f9-fe0578aa0db2 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:39:17.014287 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85350f3f-472b-485e-be4b-d1ed081fe090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:39:17.015168 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85350f3f-472b-485e-be4b-d1ed081fe090 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:39:17.015452 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85350f3f-472b-485e-be4b-d1ed081fe090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:39:17.015730 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85350f3f-472b-485e-be4b-d1ed081fe090 admin admin] PATH_INFO: `/v3/projects/8aa7b286c9d54c649a160ecf31239d4a/users/7e8e8a1fd4b0464b85644b9737c1d18b/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:39:17.030509 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85350f3f-472b-485e-be4b-d1ed081fe090 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8aa7b286c9d54c649a160ecf31239d4a, user_id=7e8e8a1fd4b0464b85644b9737c1d18b, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:39:17.031530 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: WARNING py.warnings [None req-85350f3f-472b-485e-be4b-d1ed081fe090 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 16:39:17.031530 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: warnings.warn(msg) May 19 16:39:17.031530 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: May 19 16:39:17.033492 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85350f3f-472b-485e-be4b-d1ed081fe090 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:39:17.047133 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 741/1501] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:39:16 2022] PUT /identity/v3/projects/8aa7b286c9d54c649a160ecf31239d4a/users/7e8e8a1fd4b0464b85644b9737c1d18b/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:39:17.058374 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5807a109-7236-4de2-a19b-5220c095b3f3 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:39:17.058671 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5807a109-7236-4de2-a19b-5220c095b3f3 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:39:17.058956 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5807a109-7236-4de2-a19b-5220c095b3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:39:17.084948 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-5807a109-7236-4de2-a19b-5220c095b3f3 None None] MFA Rules not processed for user `7e8e8a1fd4b0464b85644b9737c1d18b`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:39:17.086205 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-5807a109-7236-4de2-a19b-5220c095b3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:39:17.150264 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 761/1502] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:39:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:39:17.164394 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5973be62-cfd5-4ef9-b5f1-c7b056264a3e None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:39:17.164689 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5973be62-cfd5-4ef9-b5f1-c7b056264a3e None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:39:17.164976 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5973be62-cfd5-4ef9-b5f1-c7b056264a3e None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:39:17.166396 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 742/1503] 10.222.0.36 () {54 vars in 963 bytes} [Thu May 19 16:39:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:39:17.171294 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05bf99-22e5-49d5-ab54-8f53eaebf3ba None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:39:17.171593 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05bf99-22e5-49d5-ab54-8f53eaebf3ba None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:39:17.171885 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05bf99-22e5-49d5-ab54-8f53eaebf3ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:39:17.190177 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-3d05bf99-22e5-49d5-ab54-8f53eaebf3ba None None] MFA Rules not processed for user `2cc96d859a344042a4251a4104419855`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:39:17.191337 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-3d05bf99-22e5-49d5-ab54-8f53eaebf3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:39:17.234976 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:39:17.243101 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 762/1504] 10.222.0.36 () {58 vars in 1060 bytes} [Thu May 19 16:39:17 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:39:17.250734 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5973be62-cfd5-4ef9-b5f1-c7b056264a3e None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:39:17.261598 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5973be62-cfd5-4ef9-b5f1-c7b056264a3e None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:39:17.265667 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf218fa9-d4ca-4f7d-b49d-848eff9030f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:39:17.266338 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf218fa9-d4ca-4f7d-b49d-848eff9030f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:39:17.266623 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf218fa9-d4ca-4f7d-b49d-848eff9030f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:39:17.266906 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf218fa9-d4ca-4f7d-b49d-848eff9030f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:39:17.275447 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf218fa9-d4ca-4f7d-b49d-848eff9030f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:39:17.276895 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf218fa9-d4ca-4f7d-b49d-848eff9030f1 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:39:17.286813 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 743/1505] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:39:17 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:39:17.330196 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a2fe46-888d-40d7-8349-d040e158b240 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:39:17.330493 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a2fe46-888d-40d7-8349-d040e158b240 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:39:17.330782 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a2fe46-888d-40d7-8349-d040e158b240 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:39:17.332110 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 763/1506] 10.222.0.36 () {52 vars in 829 bytes} [Thu May 19 16:39:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:39:17.336439 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf218fa9-d4ca-4f7d-b49d-848eff9030f1 service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:39:17.399813 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf218fa9-d4ca-4f7d-b49d-848eff9030f1 service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:39:17.404987 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed80362-6714-4ea3-8cdb-f0a0b589252d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:39:17.405934 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed80362-6714-4ea3-8cdb-f0a0b589252d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:39:17.406236 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed80362-6714-4ea3-8cdb-f0a0b589252d admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:39:17.406499 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed80362-6714-4ea3-8cdb-f0a0b589252d admin admin] PATH_INFO: `/v3/users/7e8e8a1fd4b0464b85644b9737c1d18b` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:39:17.407723 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed80362-6714-4ea3-8cdb-f0a0b589252d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e8e8a1fd4b0464b85644b9737c1d18b)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:39:17.408749 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: WARNING py.warnings [None req-0ed80362-6714-4ea3-8cdb-f0a0b589252d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 16:39:17.408749 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: warnings.warn(msg) May 19 16:39:17.408749 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: May 19 16:39:17.409999 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed80362-6714-4ea3-8cdb-f0a0b589252d admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:39:17.476590 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0ed80362-6714-4ea3-8cdb-f0a0b589252d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e8e8a1fd4b0464b85644b9737c1d18b'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:39:17.477312 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0ed80362-6714-4ea3-8cdb-f0a0b589252d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e8e8a1fd4b0464b85644b9737c1d18b'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:39:17.489754 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0ed80362-6714-4ea3-8cdb-f0a0b589252d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e8e8a1fd4b0464b85644b9737c1d18b'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:39:17.507966 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0ed80362-6714-4ea3-8cdb-f0a0b589252d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e8e8a1fd4b0464b85644b9737c1d18b'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:39:17.508633 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0ed80362-6714-4ea3-8cdb-f0a0b589252d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e8e8a1fd4b0464b85644b9737c1d18b'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:39:17.518149 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:39:17.529757 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 744/1507] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:39:17 2022] DELETE /identity/v3/users/7e8e8a1fd4b0464b85644b9737c1d18b => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:39:17.534277 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef110b6-7673-4a6d-b1f0-493b89777e2a None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:39:17.534655 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef110b6-7673-4a6d-b1f0-493b89777e2a None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:39:17.534938 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef110b6-7673-4a6d-b1f0-493b89777e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:39:17.553290 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-0ef110b6-7673-4a6d-b1f0-493b89777e2a None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:39:17.554569 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-0ef110b6-7673-4a6d-b1f0-493b89777e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:39:17.613142 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 764/1508] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:39:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:39:17.625019 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed80362-6714-4ea3-8cdb-f0a0b589252d admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:39:17.626493 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-0ed80362-6714-4ea3-8cdb-f0a0b589252d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:39:17.688062 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed80362-6714-4ea3-8cdb-f0a0b589252d admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:39:17.691942 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6382a0-2203-49a1-9c41-fb082bf107eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:39:17.692605 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6382a0-2203-49a1-9c41-fb082bf107eb service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:39:17.692896 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6382a0-2203-49a1-9c41-fb082bf107eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:39:17.693209 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6382a0-2203-49a1-9c41-fb082bf107eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:39:17.703348 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6382a0-2203-49a1-9c41-fb082bf107eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:39:17.704825 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6382a0-2203-49a1-9c41-fb082bf107eb service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:39:17.715038 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 745/1509] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:39:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:39:18.090629 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef110b6-7673-4a6d-b1f0-493b89777e2a None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:39:18.092126 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-0ef110b6-7673-4a6d-b1f0-493b89777e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:39:18.107845 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef110b6-7673-4a6d-b1f0-493b89777e2a None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:39:18.112887 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4dfffd-950d-44f6-a993-1c7728e7c03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:39:18.114493 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4dfffd-950d-44f6-a993-1c7728e7c03b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:39:18.114744 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4dfffd-950d-44f6-a993-1c7728e7c03b admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:39:18.115024 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4dfffd-950d-44f6-a993-1c7728e7c03b admin admin] PATH_INFO: `/v3/projects/8aa7b286c9d54c649a160ecf31239d4a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:39:18.116282 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4dfffd-950d-44f6-a993-1c7728e7c03b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8aa7b286c9d54c649a160ecf31239d4a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:39:18.117370 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: WARNING py.warnings [None req-da4dfffd-950d-44f6-a993-1c7728e7c03b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 16:39:18.117370 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: warnings.warn(msg) May 19 16:39:18.117370 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: May 19 16:39:18.118589 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4dfffd-950d-44f6-a993-1c7728e7c03b admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:39:18.196830 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-da4dfffd-950d-44f6-a993-1c7728e7c03b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8aa7b286c9d54c649a160ecf31239d4a', 'request_id': 'req-da4dfffd-950d-44f6-a993-1c7728e7c03b'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:39:18.203400 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-da4dfffd-950d-44f6-a993-1c7728e7c03b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8aa7b286c9d54c649a160ecf31239d4a', 'request_id': 'req-da4dfffd-950d-44f6-a993-1c7728e7c03b'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:39:18.218920 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-da4dfffd-950d-44f6-a993-1c7728e7c03b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8aa7b286c9d54c649a160ecf31239d4a', 'request_id': 'req-da4dfffd-950d-44f6-a993-1c7728e7c03b'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:39:18.236955 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-da4dfffd-950d-44f6-a993-1c7728e7c03b admin admin] The token cache is being invalidate because project 8aa7b286c9d54c649a160ecf31239d4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:39:18.237278 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-da4dfffd-950d-44f6-a993-1c7728e7c03b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:39:18.237966 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-da4dfffd-950d-44f6-a993-1c7728e7c03b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:39:18.239739 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 765/1510] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:39:18 2022] DELETE /identity/v3/projects/8aa7b286c9d54c649a160ecf31239d4a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:39:27.710481 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=29 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:39:28.234377 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:40:58.388052 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9132302b-4ba1-4902-a892-c0187ade7750 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:40:58.388654 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9132302b-4ba1-4902-a892-c0187ade7750 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:40:58.388654 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9132302b-4ba1-4902-a892-c0187ade7750 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:40:58.390045 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 746/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 16:40:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:40:58.395474 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863869c-619b-44e3-8cee-3d1a3b45b56c None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:40:58.395923 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863869c-619b-44e3-8cee-3d1a3b45b56c None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:40:58.396084 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863869c-619b-44e3-8cee-3d1a3b45b56c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:40:58.414749 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-2863869c-619b-44e3-8cee-3d1a3b45b56c None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:40:58.415979 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-2863869c-619b-44e3-8cee-3d1a3b45b56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:40:58.475391 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 766/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 16:40:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:40:58.645117 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9132302b-4ba1-4902-a892-c0187ade7750 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:40:58.647086 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-9132302b-4ba1-4902-a892-c0187ade7750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:40:58.723078 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9132302b-4ba1-4902-a892-c0187ade7750 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:40:58.728451 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce01ba2-f6f2-4aef-8e5f-9a6b97d2b420 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:40:58.729105 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce01ba2-f6f2-4aef-8e5f-9a6b97d2b420 service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:40:58.729418 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce01ba2-f6f2-4aef-8e5f-9a6b97d2b420 service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:40:58.729695 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce01ba2-f6f2-4aef-8e5f-9a6b97d2b420 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:40:58.738391 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce01ba2-f6f2-4aef-8e5f-9a6b97d2b420 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:40:58.739999 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce01ba2-f6f2-4aef-8e5f-9a6b97d2b420 service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:40:58.749596 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 747/1513] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:40:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:40:59.657057 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87745b62-8266-40cd-bf06-4ab2a921b342 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:40:59.657645 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87745b62-8266-40cd-bf06-4ab2a921b342 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:40:59.657750 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87745b62-8266-40cd-bf06-4ab2a921b342 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:40:59.659447 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 767/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 16:40:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:40:59.664394 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8caa16b-54a9-44cf-81e9-856561fd8850 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:40:59.664831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8caa16b-54a9-44cf-81e9-856561fd8850 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:40:59.664954 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8caa16b-54a9-44cf-81e9-856561fd8850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:40:59.682849 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-c8caa16b-54a9-44cf-81e9-856561fd8850 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:40:59.684056 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-c8caa16b-54a9-44cf-81e9-856561fd8850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:40:59.694335 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 748/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 16:40:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:40:59.843966 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87745b62-8266-40cd-bf06-4ab2a921b342 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:40:59.857906 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87745b62-8266-40cd-bf06-4ab2a921b342 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:40:59.863857 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f385f2e-f6eb-415d-9055-5bb15bbffe77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:40:59.864488 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f385f2e-f6eb-415d-9055-5bb15bbffe77 service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:40:59.864769 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f385f2e-f6eb-415d-9055-5bb15bbffe77 service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:40:59.865046 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f385f2e-f6eb-415d-9055-5bb15bbffe77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:40:59.873012 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f385f2e-f6eb-415d-9055-5bb15bbffe77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:40:59.874653 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f385f2e-f6eb-415d-9055-5bb15bbffe77 service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:40:59.884370 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 768/1516] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:40:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:41:00.107182 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa77288-aa41-4ffc-b3f3-fe0cebf6bc11 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:41:00.107504 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa77288-aa41-4ffc-b3f3-fe0cebf6bc11 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:41:00.108005 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa77288-aa41-4ffc-b3f3-fe0cebf6bc11 None None] PATH_INFO: `/v3` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:41:00.109314 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 749/1517] 10.222.0.48 () {54 vars in 925 bytes} [Thu May 19 16:41:00 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:41:00.116099 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37c3f1-381b-4e6d-82fe-e43343d51ae4 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:41:00.116390 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37c3f1-381b-4e6d-82fe-e43343d51ae4 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:41:00.116673 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37c3f1-381b-4e6d-82fe-e43343d51ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:41:00.135369 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-4a37c3f1-381b-4e6d-82fe-e43343d51ae4 None None] MFA Rules not processed for user `c6e158e5c87f4954aeb297f42dabdc16`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:41:00.136568 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-4a37c3f1-381b-4e6d-82fe-e43343d51ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:41:00.197880 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 769/1518] 10.222.0.48 () {58 vars in 1015 bytes} [Thu May 19 16:41:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:41:04.139870 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724199c-d462-43d7-80de-376a8af76546 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:41:04.140450 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724199c-d462-43d7-80de-376a8af76546 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:41:04.140450 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724199c-d462-43d7-80de-376a8af76546 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:41:04.141877 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 750/1519] 10.222.0.36 () {54 vars in 960 bytes} [Thu May 19 16:41:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:41:04.147043 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0c65f-c578-4bab-92ce-feef4134cc2c None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:41:04.147554 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0c65f-c578-4bab-92ce-feef4134cc2c None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:41:04.147652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0c65f-c578-4bab-92ce-feef4134cc2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:41:04.167047 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-bcc0c65f-c578-4bab-92ce-feef4134cc2c None None] MFA Rules not processed for user `c6e158e5c87f4954aeb297f42dabdc16`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:41:04.168289 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-bcc0c65f-c578-4bab-92ce-feef4134cc2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:41:04.177627 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 770/1520] 10.222.0.36 () {58 vars in 1057 bytes} [Thu May 19 16:41:04 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:41:04.185320 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8724199c-d462-43d7-80de-376a8af76546 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:41:04.199283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8724199c-d462-43d7-80de-376a8af76546 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:41:04.205064 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbbb01e-dd1f-4b3a-a119-2d64f467d365 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e158e5c87f4954aeb297f42dabdc16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:41:04.205746 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbbb01e-dd1f-4b3a-a119-2d64f467d365 service placement] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:41:04.206032 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbbb01e-dd1f-4b3a-a119-2d64f467d365 service placement] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:41:04.206310 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbbb01e-dd1f-4b3a-a119-2d64f467d365 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:41:04.215250 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbbb01e-dd1f-4b3a-a119-2d64f467d365 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:41:04.216865 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbbb01e-dd1f-4b3a-a119-2d64f467d365 service placement] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:41:04.227712 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 751/1521] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:41:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 16:41:11.201298 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e486c99-72a8-4155-a2a3-1fc6df19f920 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:41:11.201834 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e486c99-72a8-4155-a2a3-1fc6df19f920 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:41:11.201834 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e486c99-72a8-4155-a2a3-1fc6df19f920 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:41:11.203184 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 771/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 16:41:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:41:11.208048 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13b6ee-413d-490a-9ad9-0a6b9efde597 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:41:11.208494 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13b6ee-413d-490a-9ad9-0a6b9efde597 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:41:11.208602 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13b6ee-413d-490a-9ad9-0a6b9efde597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:41:11.226534 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-6e13b6ee-413d-490a-9ad9-0a6b9efde597 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:41:11.227734 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-6e13b6ee-413d-490a-9ad9-0a6b9efde597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:41:11.235536 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 752/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 16:41:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:41:11.387809 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e486c99-72a8-4155-a2a3-1fc6df19f920 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:41:11.402046 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e486c99-72a8-4155-a2a3-1fc6df19f920 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:41:11.407701 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533db92c-debe-424e-90ba-60a5600344c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:41:11.408347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533db92c-debe-424e-90ba-60a5600344c0 service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:41:11.408641 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533db92c-debe-424e-90ba-60a5600344c0 service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:41:11.408927 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533db92c-debe-424e-90ba-60a5600344c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:41:11.417976 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533db92c-debe-424e-90ba-60a5600344c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:41:11.419624 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533db92c-debe-424e-90ba-60a5600344c0 service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:41:11.429819 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 772/1524] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:41:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 16:41:12.333388 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cfc9ab-c080-40dd-a000-230d30ebc729 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:41:12.333932 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cfc9ab-c080-40dd-a000-230d30ebc729 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:41:12.333932 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cfc9ab-c080-40dd-a000-230d30ebc729 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:41:12.335465 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 753/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 16:41:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:41:12.340186 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a085b-055f-4ba6-8b04-da3dc839dc46 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:41:12.340575 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a085b-055f-4ba6-8b04-da3dc839dc46 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:41:12.340757 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a085b-055f-4ba6-8b04-da3dc839dc46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:41:12.358965 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-1b4a085b-055f-4ba6-8b04-da3dc839dc46 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:41:12.360186 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-1b4a085b-055f-4ba6-8b04-da3dc839dc46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:41:12.368607 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 773/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 16:41:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:41:12.520885 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cfc9ab-c080-40dd-a000-230d30ebc729 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:41:12.535779 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cfc9ab-c080-40dd-a000-230d30ebc729 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:41:12.541073 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48339a71-fd04-4639-90a2-457ce73d27d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:41:12.541736 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48339a71-fd04-4639-90a2-457ce73d27d2 service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:41:12.542021 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48339a71-fd04-4639-90a2-457ce73d27d2 service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:41:12.542296 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48339a71-fd04-4639-90a2-457ce73d27d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:41:12.550244 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48339a71-fd04-4639-90a2-457ce73d27d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:41:12.551812 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48339a71-fd04-4639-90a2-457ce73d27d2 service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:41:12.561530 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 754/1527] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:41:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:41:16.621533 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af74cc-32b9-4687-b03a-32d4f35c495a None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:41:16.622116 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af74cc-32b9-4687-b03a-32d4f35c495a None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:41:16.622116 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af74cc-32b9-4687-b03a-32d4f35c495a None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:41:16.623453 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 774/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 16:41:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:41:16.628639 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31128518-afb2-4a7f-ba82-cd3aa48cac52 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:41:16.629075 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31128518-afb2-4a7f-ba82-cd3aa48cac52 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:41:16.629262 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31128518-afb2-4a7f-ba82-cd3aa48cac52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:41:16.647653 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-31128518-afb2-4a7f-ba82-cd3aa48cac52 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:41:16.648878 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-31128518-afb2-4a7f-ba82-cd3aa48cac52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:41:16.656777 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 755/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 16:41:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:41:16.693289 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af74cc-32b9-4687-b03a-32d4f35c495a None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:41:16.695281 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-70af74cc-32b9-4687-b03a-32d4f35c495a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:41:16.753110 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af74cc-32b9-4687-b03a-32d4f35c495a None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:41:16.756591 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2934dc-88e0-46d6-aae6-4709f4331810 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:41:16.757264 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2934dc-88e0-46d6-aae6-4709f4331810 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:41:16.757550 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2934dc-88e0-46d6-aae6-4709f4331810 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:41:16.757900 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2934dc-88e0-46d6-aae6-4709f4331810 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:41:16.766982 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2934dc-88e0-46d6-aae6-4709f4331810 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:41:16.768406 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2934dc-88e0-46d6-aae6-4709f4331810 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:41:16.773203 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:41:16.778174 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 775/1530] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:41:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:41:18.990267 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a7138-1506-430d-a4a6-8d970a9fea97 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:41:18.990841 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a7138-1506-430d-a4a6-8d970a9fea97 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:41:18.990841 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a7138-1506-430d-a4a6-8d970a9fea97 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:41:18.992221 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 756/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 16:41:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:41:18.997682 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985b2b2-b1fb-4a03-a807-2c26e6019711 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:41:18.998150 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985b2b2-b1fb-4a03-a807-2c26e6019711 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:41:18.998263 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985b2b2-b1fb-4a03-a807-2c26e6019711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:41:19.018303 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-2985b2b2-b1fb-4a03-a807-2c26e6019711 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:41:19.019558 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-2985b2b2-b1fb-4a03-a807-2c26e6019711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:41:19.027411 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 776/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 16:41:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:41:19.073676 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a7138-1506-430d-a4a6-8d970a9fea97 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:41:19.089486 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a7138-1506-430d-a4a6-8d970a9fea97 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:41:19.093122 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9e3e30-3b4f-4aa8-ba65-d23fd3e69ad7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:41:19.093854 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e3e30-3b4f-4aa8-ba65-d23fd3e69ad7 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:41:19.094138 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e3e30-3b4f-4aa8-ba65-d23fd3e69ad7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:41:19.094420 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e3e30-3b4f-4aa8-ba65-d23fd3e69ad7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:41:19.106077 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9e3e30-3b4f-4aa8-ba65-d23fd3e69ad7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:41:19.108008 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9e3e30-3b4f-4aa8-ba65-d23fd3e69ad7 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:41:19.119976 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 757/1533] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:41:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 16:41:22.857796 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b7602-5fa0-4733-8c04-d8560f2e5ac4 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:41:22.858383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b7602-5fa0-4733-8c04-d8560f2e5ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:41:22.858383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b7602-5fa0-4733-8c04-d8560f2e5ac4 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:41:22.859757 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 777/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 16:41:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:41:22.865448 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ddc88-048f-4e9b-85c2-904c6dea0c7d None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:41:22.865849 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ddc88-048f-4e9b-85c2-904c6dea0c7d None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:41:22.866013 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ddc88-048f-4e9b-85c2-904c6dea0c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:41:22.884464 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-1b6ddc88-048f-4e9b-85c2-904c6dea0c7d None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:41:22.885669 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-1b6ddc88-048f-4e9b-85c2-904c6dea0c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:41:22.893414 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 758/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 16:41:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:41:23.192954 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b7602-5fa0-4733-8c04-d8560f2e5ac4 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:41:23.194696 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-6d5b7602-5fa0-4733-8c04-d8560f2e5ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:41:23.250203 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b7602-5fa0-4733-8c04-d8560f2e5ac4 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:41:23.254439 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3888491e-6785-4dcf-8288-fe54b246838a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:41:23.255081 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888491e-6785-4dcf-8288-fe54b246838a service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:41:23.255369 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888491e-6785-4dcf-8288-fe54b246838a service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:41:23.255647 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888491e-6785-4dcf-8288-fe54b246838a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:41:23.264708 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3888491e-6785-4dcf-8288-fe54b246838a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:41:23.266177 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3888491e-6785-4dcf-8288-fe54b246838a service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:41:23.275364 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 778/1536] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:41:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:41:25.571268 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f6d971-ec9e-480b-b7b0-ac4208f8434e None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:41:25.571852 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f6d971-ec9e-480b-b7b0-ac4208f8434e None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:41:25.571852 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f6d971-ec9e-480b-b7b0-ac4208f8434e None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:41:25.573333 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 759/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 16:41:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 16:41:25.578455 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f3367-63d4-4577-9df1-ffa1db9a1f01 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:41:25.578903 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f3367-63d4-4577-9df1-ffa1db9a1f01 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:41:25.579015 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f3367-63d4-4577-9df1-ffa1db9a1f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:41:25.597376 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-b89f3367-63d4-4577-9df1-ffa1db9a1f01 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:41:25.598634 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b89f3367-63d4-4577-9df1-ffa1db9a1f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:41:25.606379 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 779/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 16:41:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:41:25.876998 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f6d971-ec9e-480b-b7b0-ac4208f8434e None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:41:25.878727 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-45f6d971-ec9e-480b-b7b0-ac4208f8434e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:41:25.891972 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f6d971-ec9e-480b-b7b0-ac4208f8434e None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:41:25.897604 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ab412-7844-48d3-9445-497cd39c20fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:41:25.898266 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ab412-7844-48d3-9445-497cd39c20fb service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:41:25.898579 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ab412-7844-48d3-9445-497cd39c20fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:41:25.898858 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ab412-7844-48d3-9445-497cd39c20fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:41:25.908470 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5ab412-7844-48d3-9445-497cd39c20fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:41:25.910541 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5ab412-7844-48d3-9445-497cd39c20fb service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:41:25.920094 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 760/1539] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:41:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:41:35.592153 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=32 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:41:35.915782 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=70 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:42:56.079071 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5227f6e-cc52-4b05-ac8f-129b204a77ad None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:56.079720 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5227f6e-cc52-4b05-ac8f-129b204a77ad None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:56.079720 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5227f6e-cc52-4b05-ac8f-129b204a77ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:56.099318 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f8fd8-4090-4293-b907-208321d2b080 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:56.100386 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f8fd8-4090-4293-b907-208321d2b080 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:56.104596 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f8fd8-4090-4293-b907-208321d2b080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:56.141198 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-b5227f6e-cc52-4b05-ac8f-129b204a77ad None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:42:56.141198 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b5227f6e-cc52-4b05-ac8f-129b204a77ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:42:56.150672 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-512f8fd8-4090-4293-b907-208321d2b080 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:42:56.152719 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-512f8fd8-4090-4293-b907-208321d2b080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:42:56.163043 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 780/1540] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:42:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:42:56.171054 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 761/1541] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:42:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:42:56.175266 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc2b25-640f-4657-a7c9-5fe2bd9e751c None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:56.175581 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc2b25-640f-4657-a7c9-5fe2bd9e751c None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:56.176004 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc2b25-640f-4657-a7c9-5fe2bd9e751c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:56.177804 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100475ba-fe11-4d62-bd6b-8cf4cc3b544d None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:56.178101 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100475ba-fe11-4d62-bd6b-8cf4cc3b544d None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:56.178382 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100475ba-fe11-4d62-bd6b-8cf4cc3b544d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:56.214225 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-100475ba-fe11-4d62-bd6b-8cf4cc3b544d None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:42:56.215392 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-83bc2b25-640f-4657-a7c9-5fe2bd9e751c None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:42:56.215827 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-100475ba-fe11-4d62-bd6b-8cf4cc3b544d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:42:56.217118 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-83bc2b25-640f-4657-a7c9-5fe2bd9e751c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:42:56.232519 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 781/1542] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:42:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:42:56.233794 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 762/1543] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:42:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:42:56.234828 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79367e40-8c18-4a49-b96d-62f34b13739f None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:56.235083 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f54595-3fca-4e67-ba88-2520664c3e64 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:56.235572 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79367e40-8c18-4a49-b96d-62f34b13739f None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:56.235699 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f54595-3fca-4e67-ba88-2520664c3e64 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:56.235962 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f54595-3fca-4e67-ba88-2520664c3e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:56.236957 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79367e40-8c18-4a49-b96d-62f34b13739f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:56.276558 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-c4f54595-3fca-4e67-ba88-2520664c3e64 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:42:56.277539 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-79367e40-8c18-4a49-b96d-62f34b13739f None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:42:56.277899 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-c4f54595-3fca-4e67-ba88-2520664c3e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:42:56.279245 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-79367e40-8c18-4a49-b96d-62f34b13739f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:42:56.293580 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 763/1544] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:42:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:42:56.294445 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f54595-3fca-4e67-ba88-2520664c3e64 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:56.297503 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 782/1545] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:42:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:42:56.299754 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb633ba-8dfe-47bc-93bf-1b9fc61a6444 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:56.300310 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb633ba-8dfe-47bc-93bf-1b9fc61a6444 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:56.300891 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb633ba-8dfe-47bc-93bf-1b9fc61a6444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:56.320055 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f54595-3fca-4e67-ba88-2520664c3e64 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:56.327258 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd94931a-30b5-4bc2-b9e0-6f2d138fc44a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:56.327943 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94931a-30b5-4bc2-b9e0-6f2d138fc44a admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:56.328227 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94931a-30b5-4bc2-b9e0-6f2d138fc44a admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:56.328503 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94931a-30b5-4bc2-b9e0-6f2d138fc44a admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:56.329498 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd94931a-30b5-4bc2-b9e0-6f2d138fc44a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:42:56.329978 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd94931a-30b5-4bc2-b9e0-6f2d138fc44a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:56.331069 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: WARNING py.warnings [None req-cd94931a-30b5-4bc2-b9e0-6f2d138fc44a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 19 16:42:56.331069 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: warnings.warn(msg) May 19 16:42:56.331069 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: May 19 16:42:56.331983 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd94931a-30b5-4bc2-b9e0-6f2d138fc44a admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:56.344924 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-5fb633ba-8dfe-47bc-93bf-1b9fc61a6444 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:42:56.348162 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-5fb633ba-8dfe-47bc-93bf-1b9fc61a6444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:42:56.357096 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 764/1546] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:42:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:42:56.358742 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd94931a-30b5-4bc2-b9e0-6f2d138fc44a admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:56.368076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 783/1547] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:42:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:42:56.369492 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2a7aa-59b1-442a-9a7d-58a6107b8e2d None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:56.369793 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2a7aa-59b1-442a-9a7d-58a6107b8e2d None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:56.370085 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2a7aa-59b1-442a-9a7d-58a6107b8e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:56.376085 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd94931a-30b5-4bc2-b9e0-6f2d138fc44a admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:56.383440 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423045d0-ec1c-4c56-a011-3d2c15a8bbfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:56.384121 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423045d0-ec1c-4c56-a011-3d2c15a8bbfd admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:56.384956 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423045d0-ec1c-4c56-a011-3d2c15a8bbfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:56.385281 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423045d0-ec1c-4c56-a011-3d2c15a8bbfd admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:56.386021 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423045d0-ec1c-4c56-a011-3d2c15a8bbfd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:42:56.386332 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423045d0-ec1c-4c56-a011-3d2c15a8bbfd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:56.388281 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423045d0-ec1c-4c56-a011-3d2c15a8bbfd admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:56.391105 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-c7c2a7aa-59b1-442a-9a7d-58a6107b8e2d None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:42:56.392339 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-c7c2a7aa-59b1-442a-9a7d-58a6107b8e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:42:56.401275 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 784/1548] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:42:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:42:56.402406 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df6c127-3c18-45f8-89b0-1092514811cd None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:56.402704 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df6c127-3c18-45f8-89b0-1092514811cd None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:56.402993 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df6c127-3c18-45f8-89b0-1092514811cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:56.403812 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 765/1549] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:42:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:42:56.410519 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9503dbf-a86a-4ba3-a09e-00361728d59b None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:56.410946 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9503dbf-a86a-4ba3-a09e-00361728d59b None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:56.412435 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9503dbf-a86a-4ba3-a09e-00361728d59b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:56.428638 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-6df6c127-3c18-45f8-89b0-1092514811cd None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:42:56.429888 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-6df6c127-3c18-45f8-89b0-1092514811cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:42:56.436261 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-e9503dbf-a86a-4ba3-a09e-00361728d59b None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:42:56.437492 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-e9503dbf-a86a-4ba3-a09e-00361728d59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:42:56.437789 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 785/1550] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:42:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:42:56.438554 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df6c127-3c18-45f8-89b0-1092514811cd None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:56.449123 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 766/1551] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:42:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:42:56.452314 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9503dbf-a86a-4ba3-a09e-00361728d59b None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:56.455291 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df6c127-3c18-45f8-89b0-1092514811cd None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:56.462872 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9503dbf-a86a-4ba3-a09e-00361728d59b None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:56.469017 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5181b23e-ee22-4b09-92b2-73f67bde52ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:56.469725 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181b23e-ee22-4b09-92b2-73f67bde52ed admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:56.470015 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181b23e-ee22-4b09-92b2-73f67bde52ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:56.470294 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181b23e-ee22-4b09-92b2-73f67bde52ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:56.471035 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5181b23e-ee22-4b09-92b2-73f67bde52ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:56.471409 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85af82c-faf8-46bd-917d-6e06357cbdaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:56.472086 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85af82c-faf8-46bd-917d-6e06357cbdaa admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:56.472191 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: WARNING py.warnings [None req-5181b23e-ee22-4b09-92b2-73f67bde52ed admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 16:42:56.472191 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: warnings.warn(msg) May 19 16:42:56.472191 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: May 19 16:42:56.472445 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85af82c-faf8-46bd-917d-6e06357cbdaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:56.472650 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85af82c-faf8-46bd-917d-6e06357cbdaa admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:56.473326 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5181b23e-ee22-4b09-92b2-73f67bde52ed admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:56.473426 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85af82c-faf8-46bd-917d-6e06357cbdaa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:42:56.473726 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85af82c-faf8-46bd-917d-6e06357cbdaa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:56.475349 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85af82c-faf8-46bd-917d-6e06357cbdaa admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:56.493624 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 767/1552] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:42:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:42:56.494846 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85af82c-faf8-46bd-917d-6e06357cbdaa admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:56.507585 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 786/1553] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:42:56 2022] POST /identity/v3/projects => generated 377 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:42:56.508753 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5181b23e-ee22-4b09-92b2-73f67bde52ed admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:56.511739 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85af82c-faf8-46bd-917d-6e06357cbdaa admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:56.567562 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81635740-9587-4351-81ea-4086b3b4e0bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:56.568680 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81635740-9587-4351-81ea-4086b3b4e0bb admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:56.568680 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81635740-9587-4351-81ea-4086b3b4e0bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:56.568890 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81635740-9587-4351-81ea-4086b3b4e0bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:56.569618 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81635740-9587-4351-81ea-4086b3b4e0bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:42:56.569872 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81635740-9587-4351-81ea-4086b3b4e0bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:56.571462 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81635740-9587-4351-81ea-4086b3b4e0bb admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:56.572146 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5181b23e-ee22-4b09-92b2-73f67bde52ed admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:56.576714 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb646db-511b-4a98-97d4-b34b3b4452f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:56.577410 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb646db-511b-4a98-97d4-b34b3b4452f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:56.577697 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb646db-511b-4a98-97d4-b34b3b4452f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:56.577979 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb646db-511b-4a98-97d4-b34b3b4452f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:56.578674 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb646db-511b-4a98-97d4-b34b3b4452f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:56.580677 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb646db-511b-4a98-97d4-b34b3b4452f1 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:56.584583 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 768/1554] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:42:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:42:56.584583 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52e143-c4b6-4fe7-b6d5-c4c0fae5c4b5 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:56.584583 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52e143-c4b6-4fe7-b6d5-c4c0fae5c4b5 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:56.584583 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52e143-c4b6-4fe7-b6d5-c4c0fae5c4b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:56.605091 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-2f52e143-c4b6-4fe7-b6d5-c4c0fae5c4b5 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:42:56.606449 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-2f52e143-c4b6-4fe7-b6d5-c4c0fae5c4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:42:56.607647 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 787/1555] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:42:56 2022] POST /identity/v3/projects => generated 393 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:42:56.608946 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fac4a6-1bf0-4cd9-b45a-5755069242ef None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:56.609270 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fac4a6-1bf0-4cd9-b45a-5755069242ef None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:56.609772 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fac4a6-1bf0-4cd9-b45a-5755069242ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:56.628420 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-b9fac4a6-1bf0-4cd9-b45a-5755069242ef None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:42:56.629693 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b9fac4a6-1bf0-4cd9-b45a-5755069242ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:42:56.680964 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 769/1556] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:42:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:42:56.682307 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3f9abe-709d-4076-876e-84c045985778 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:56.682608 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3f9abe-709d-4076-876e-84c045985778 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:56.682892 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3f9abe-709d-4076-876e-84c045985778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:56.690294 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 788/1557] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:42:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:42:56.691615 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65950294-dfac-456d-b005-03675d6ac6ad None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:56.691911 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65950294-dfac-456d-b005-03675d6ac6ad None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:56.692205 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65950294-dfac-456d-b005-03675d6ac6ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:56.702138 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-3c3f9abe-709d-4076-876e-84c045985778 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:42:56.703354 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-3c3f9abe-709d-4076-876e-84c045985778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:42:56.713186 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 770/1558] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:42:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:42:56.714076 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3f9abe-709d-4076-876e-84c045985778 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:56.719173 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-65950294-dfac-456d-b005-03675d6ac6ad None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:42:56.720406 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-65950294-dfac-456d-b005-03675d6ac6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:42:56.726819 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3f9abe-709d-4076-876e-84c045985778 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:56.728844 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 789/1559] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:42:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:42:56.729643 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65950294-dfac-456d-b005-03675d6ac6ad None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:56.734779 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4c01ab-c051-41f1-a151-243b65f343e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:56.735439 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c01ab-c051-41f1-a151-243b65f343e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:56.735724 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c01ab-c051-41f1-a151-243b65f343e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:56.736001 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c01ab-c051-41f1-a151-243b65f343e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:56.737322 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4c01ab-c051-41f1-a151-243b65f343e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:56.739277 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4c01ab-c051-41f1-a151-243b65f343e2 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:56.741827 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65950294-dfac-456d-b005-03675d6ac6ad None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:56.746867 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8defa0f9-8e80-4166-9046-aaab7778e482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:56.747527 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8defa0f9-8e80-4166-9046-aaab7778e482 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:56.747809 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8defa0f9-8e80-4166-9046-aaab7778e482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:56.748084 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8defa0f9-8e80-4166-9046-aaab7778e482 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:56.748774 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8defa0f9-8e80-4166-9046-aaab7778e482 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:56.750782 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8defa0f9-8e80-4166-9046-aaab7778e482 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:56.761318 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:42:56.774771 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:42:56.779538 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 771/1560] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:42:56 2022] POST /identity/v3/projects => generated 385 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:42:56.780297 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 790/1561] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:42:56 2022] POST /identity/v3/users => generated 289 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:42:56.781674 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8defa0f9-8e80-4166-9046-aaab7778e482 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:56.782741 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4c01ab-c051-41f1-a151-243b65f343e2 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:56.801462 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8defa0f9-8e80-4166-9046-aaab7778e482 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:56.863823 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d43232-7069-4710-97ac-42760ac9557a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:56.864637 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d43232-7069-4710-97ac-42760ac9557a admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:56.864992 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d43232-7069-4710-97ac-42760ac9557a admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:56.865312 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d43232-7069-4710-97ac-42760ac9557a admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:56.866025 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d43232-7069-4710-97ac-42760ac9557a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:56.868013 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d43232-7069-4710-97ac-42760ac9557a admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:56.875793 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4c01ab-c051-41f1-a151-243b65f343e2 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:56.881374 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc4237a-2c13-4edc-bcea-8cb55d1917af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:56.882546 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4237a-2c13-4edc-bcea-8cb55d1917af admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:56.882762 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4237a-2c13-4edc-bcea-8cb55d1917af admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:56.882972 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4237a-2c13-4edc-bcea-8cb55d1917af admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:56.883790 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc4237a-2c13-4edc-bcea-8cb55d1917af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:56.885807 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc4237a-2c13-4edc-bcea-8cb55d1917af admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:56.899513 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 791/1562] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:42:56 2022] POST /identity/v3/users => generated 297 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:42:56.899513 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e9171-c0cf-457b-838d-8d991475abc3 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:56.900256 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e9171-c0cf-457b-838d-8d991475abc3 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:56.900618 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e9171-c0cf-457b-838d-8d991475abc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:56.919917 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 772/1563] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:42:56 2022] POST /identity/v3/projects => generated 383 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:42:56.919917 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10486daf-eead-4a4f-95c7-cf48606da8f2 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:56.919917 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10486daf-eead-4a4f-95c7-cf48606da8f2 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:56.919917 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10486daf-eead-4a4f-95c7-cf48606da8f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:56.931649 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-2c4e9171-c0cf-457b-838d-8d991475abc3 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:42:56.934206 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-2c4e9171-c0cf-457b-838d-8d991475abc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:42:56.937056 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-10486daf-eead-4a4f-95c7-cf48606da8f2 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:42:56.938288 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-10486daf-eead-4a4f-95c7-cf48606da8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:42:56.999655 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 792/1564] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:42:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:42:57.000565 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4e9171-c0cf-457b-838d-8d991475abc3 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:57.006361 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 773/1565] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:42:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:42:57.008356 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10486daf-eead-4a4f-95c7-cf48606da8f2 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:57.012043 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4e9171-c0cf-457b-838d-8d991475abc3 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:57.016688 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e933ef0-1f7a-43bc-b467-c7fec691cb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:57.017449 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e933ef0-1f7a-43bc-b467-c7fec691cb4f admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:57.018269 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e933ef0-1f7a-43bc-b467-c7fec691cb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:57.018549 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e933ef0-1f7a-43bc-b467-c7fec691cb4f admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:57.019197 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10486daf-eead-4a4f-95c7-cf48606da8f2 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:57.019313 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e933ef0-1f7a-43bc-b467-c7fec691cb4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:42:57.019602 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e933ef0-1f7a-43bc-b467-c7fec691cb4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:57.021292 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e933ef0-1f7a-43bc-b467-c7fec691cb4f admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:57.023838 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec19d397-2d0d-48f3-a90e-200892135df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:57.024509 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec19d397-2d0d-48f3-a90e-200892135df3 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:57.024794 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec19d397-2d0d-48f3-a90e-200892135df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:57.025073 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec19d397-2d0d-48f3-a90e-200892135df3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:57.025843 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec19d397-2d0d-48f3-a90e-200892135df3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:42:57.025843 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec19d397-2d0d-48f3-a90e-200892135df3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:57.027745 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec19d397-2d0d-48f3-a90e-200892135df3 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:57.034273 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 793/1566] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:42:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:42:57.035100 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e933ef0-1f7a-43bc-b467-c7fec691cb4f admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:57.041963 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 774/1567] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:42:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:42:57.043037 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec19d397-2d0d-48f3-a90e-200892135df3 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:57.044958 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e933ef0-1f7a-43bc-b467-c7fec691cb4f admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:57.053271 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d82d43-0c06-4e07-9ad2-8a28c68bcdac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:57.053963 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d82d43-0c06-4e07-9ad2-8a28c68bcdac admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:57.054241 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d82d43-0c06-4e07-9ad2-8a28c68bcdac admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:57.054519 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d82d43-0c06-4e07-9ad2-8a28c68bcdac admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:57.055225 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d82d43-0c06-4e07-9ad2-8a28c68bcdac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:57.057203 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d82d43-0c06-4e07-9ad2-8a28c68bcdac admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:57.059290 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec19d397-2d0d-48f3-a90e-200892135df3 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:57.065271 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b559d5b-70d2-4cbd-af2c-e4b723a6e782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:57.066244 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b559d5b-70d2-4cbd-af2c-e4b723a6e782 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:57.066244 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b559d5b-70d2-4cbd-af2c-e4b723a6e782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:57.066755 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b559d5b-70d2-4cbd-af2c-e4b723a6e782 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:57.067223 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b559d5b-70d2-4cbd-af2c-e4b723a6e782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:57.068837 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b559d5b-70d2-4cbd-af2c-e4b723a6e782 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:57.079087 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 794/1568] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:42:57 2022] POST /identity/v3/users => generated 293 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:42:57.080466 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d82d43-0c06-4e07-9ad2-8a28c68bcdac admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:57.084216 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 775/1569] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:42:57 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:42:57.084216 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b559d5b-70d2-4cbd-af2c-e4b723a6e782 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:57.093358 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b559d5b-70d2-4cbd-af2c-e4b723a6e782 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:57.093712 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d82d43-0c06-4e07-9ad2-8a28c68bcdac admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:57.099485 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e297c63-0a45-4472-ab70-75852c0d8e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:57.099667 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c2b45-67f9-4bd2-8f16-f51704e03a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:57.100178 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e297c63-0a45-4472-ab70-75852c0d8e02 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:57.100338 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c2b45-67f9-4bd2-8f16-f51704e03a80 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:57.100478 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e297c63-0a45-4472-ab70-75852c0d8e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:57.100594 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c2b45-67f9-4bd2-8f16-f51704e03a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:57.100695 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e297c63-0a45-4472-ab70-75852c0d8e02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:57.100876 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c2b45-67f9-4bd2-8f16-f51704e03a80 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:57.101687 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808c2b45-67f9-4bd2-8f16-f51704e03a80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:57.101866 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e297c63-0a45-4472-ab70-75852c0d8e02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:57.102666 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: WARNING py.warnings [None req-808c2b45-67f9-4bd2-8f16-f51704e03a80 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 16:42:57.102666 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: warnings.warn(msg) May 19 16:42:57.102666 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: May 19 16:42:57.103928 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808c2b45-67f9-4bd2-8f16-f51704e03a80 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:57.104162 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e297c63-0a45-4472-ab70-75852c0d8e02 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:57.117444 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 795/1570] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:42:57 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:42:57.118387 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e297c63-0a45-4472-ab70-75852c0d8e02 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:57.131433 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e297c63-0a45-4472-ab70-75852c0d8e02 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:57.131919 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 776/1571] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:42:57 2022] POST /identity/v3/users => generated 292 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:42:57.133183 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c2b45-67f9-4bd2-8f16-f51704e03a80 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:57.139204 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c3f50-62f4-4cda-b19c-27a39fec4242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:57.139906 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c3f50-62f4-4cda-b19c-27a39fec4242 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:57.140196 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c3f50-62f4-4cda-b19c-27a39fec4242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:57.140478 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c3f50-62f4-4cda-b19c-27a39fec4242 admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:57.141270 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c3f50-62f4-4cda-b19c-27a39fec4242 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:42:57.141591 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c3f50-62f4-4cda-b19c-27a39fec4242 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:57.143819 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c3f50-62f4-4cda-b19c-27a39fec4242 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:57.147793 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c2b45-67f9-4bd2-8f16-f51704e03a80 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:57.152583 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b99c1d6-338c-4897-af48-17b46e48c569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:57.153294 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b99c1d6-338c-4897-af48-17b46e48c569 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:57.153589 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b99c1d6-338c-4897-af48-17b46e48c569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:57.153871 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b99c1d6-338c-4897-af48-17b46e48c569 admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:57.154626 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b99c1d6-338c-4897-af48-17b46e48c569 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:42:57.154945 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b99c1d6-338c-4897-af48-17b46e48c569 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:57.156571 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b99c1d6-338c-4897-af48-17b46e48c569 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:57.156979 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 796/1572] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:42:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:42:57.157791 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c3f50-62f4-4cda-b19c-27a39fec4242 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:57.168334 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c3f50-62f4-4cda-b19c-27a39fec4242 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:57.174328 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec016a4-01e6-49c9-a750-1ccbce518fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:57.175022 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec016a4-01e6-49c9-a750-1ccbce518fef admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:57.175310 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec016a4-01e6-49c9-a750-1ccbce518fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:57.175586 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec016a4-01e6-49c9-a750-1ccbce518fef admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:57.176295 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec016a4-01e6-49c9-a750-1ccbce518fef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:57.178450 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec016a4-01e6-49c9-a750-1ccbce518fef admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:57.178676 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 777/1573] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:42:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:42:57.179483 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b99c1d6-338c-4897-af48-17b46e48c569 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:57.192032 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b99c1d6-338c-4897-af48-17b46e48c569 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:57.196846 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c8e1a4-4c7b-4aee-b709-a4c34d3960b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:57.197551 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c8e1a4-4c7b-4aee-b709-a4c34d3960b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:57.197846 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c8e1a4-4c7b-4aee-b709-a4c34d3960b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:57.198131 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c8e1a4-4c7b-4aee-b709-a4c34d3960b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:57.198914 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c8e1a4-4c7b-4aee-b709-a4c34d3960b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:57.201133 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c8e1a4-4c7b-4aee-b709-a4c34d3960b3 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:57.214690 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 797/1574] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:42:57 2022] POST /identity/v3/projects => generated 361 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:42:57.214690 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec016a4-01e6-49c9-a750-1ccbce518fef admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:57.227567 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:42:57.233450 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 778/1575] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:42:57 2022] POST /identity/v3/projects => generated 357 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:42:57.234222 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c8e1a4-4c7b-4aee-b709-a4c34d3960b3 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:57.246206 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:42:57.285877 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec016a4-01e6-49c9-a750-1ccbce518fef admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:57.303749 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c8e1a4-4c7b-4aee-b709-a4c34d3960b3 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:57.308460 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76402060-24a7-4102-ac4d-78e761389aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:57.309165 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76402060-24a7-4102-ac4d-78e761389aec admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:57.309461 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76402060-24a7-4102-ac4d-78e761389aec admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:57.309744 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76402060-24a7-4102-ac4d-78e761389aec admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:57.310486 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76402060-24a7-4102-ac4d-78e761389aec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:57.312212 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76402060-24a7-4102-ac4d-78e761389aec admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:57.326606 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 779/1576] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:42:57 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:42:57.326606 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76402060-24a7-4102-ac4d-78e761389aec admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:57.337235 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76402060-24a7-4102-ac4d-78e761389aec admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:57.342225 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d468efc1-926a-4caa-9e6d-98603a4c7476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:57.342903 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468efc1-926a-4caa-9e6d-98603a4c7476 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:57.343196 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468efc1-926a-4caa-9e6d-98603a4c7476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:57.343473 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468efc1-926a-4caa-9e6d-98603a4c7476 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:57.344212 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d468efc1-926a-4caa-9e6d-98603a4c7476 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:57.346414 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4dde1e-5fe7-4664-9650-b4905ae8a695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:57.346640 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d468efc1-926a-4caa-9e6d-98603a4c7476 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:57.347101 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4dde1e-5fe7-4664-9650-b4905ae8a695 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:57.347392 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4dde1e-5fe7-4664-9650-b4905ae8a695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:57.347677 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4dde1e-5fe7-4664-9650-b4905ae8a695 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:57.348426 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4dde1e-5fe7-4664-9650-b4905ae8a695 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:57.350133 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4dde1e-5fe7-4664-9650-b4905ae8a695 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:57.357680 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 780/1577] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:42:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:42:57.358764 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d468efc1-926a-4caa-9e6d-98603a4c7476 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:57.363599 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 798/1578] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:42:57 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:42:57.363599 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4dde1e-5fe7-4664-9650-b4905ae8a695 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:57.369444 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d468efc1-926a-4caa-9e6d-98603a4c7476 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:57.372002 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4dde1e-5fe7-4664-9650-b4905ae8a695 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:57.374553 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6e1396-6649-4ef3-b96f-0d6e12853814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:57.375746 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e1396-6649-4ef3-b96f-0d6e12853814 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:57.376074 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e1396-6649-4ef3-b96f-0d6e12853814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:57.376364 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e1396-6649-4ef3-b96f-0d6e12853814 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:57.376643 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccb5922-050e-44ab-aa09-8527adb524db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:57.377687 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6e1396-6649-4ef3-b96f-0d6e12853814 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:57.377864 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb5922-050e-44ab-aa09-8527adb524db admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:57.378152 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb5922-050e-44ab-aa09-8527adb524db admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:57.378426 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb5922-050e-44ab-aa09-8527adb524db admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:57.379122 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccb5922-050e-44ab-aa09-8527adb524db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:57.379318 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6e1396-6649-4ef3-b96f-0d6e12853814 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:57.381047 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccb5922-050e-44ab-aa09-8527adb524db admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:57.390730 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 781/1579] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:42:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:42:57.391643 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6e1396-6649-4ef3-b96f-0d6e12853814 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:57.402511 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6e1396-6649-4ef3-b96f-0d6e12853814 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:57.407208 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f98b59-cd6e-4f5c-9a81-9abb8faff4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:57.407903 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f98b59-cd6e-4f5c-9a81-9abb8faff4cf admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:57.408197 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f98b59-cd6e-4f5c-9a81-9abb8faff4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:57.408488 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f98b59-cd6e-4f5c-9a81-9abb8faff4cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:57.408738 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 799/1580] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:42:57 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:42:57.409592 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccb5922-050e-44ab-aa09-8527adb524db admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:57.412529 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f98b59-cd6e-4f5c-9a81-9abb8faff4cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:57.414529 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f98b59-cd6e-4f5c-9a81-9abb8faff4cf admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:57.448954 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 782/1581] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:42:57 2022] POST /identity/v3/projects => generated 389 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:42:57.450131 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f98b59-cd6e-4f5c-9a81-9abb8faff4cf admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:57.491984 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccb5922-050e-44ab-aa09-8527adb524db admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:57.533824 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f98b59-cd6e-4f5c-9a81-9abb8faff4cf admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:57.538694 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69213bf-4c74-4f29-9192-5391518f5b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:57.539397 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69213bf-4c74-4f29-9192-5391518f5b4c admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:57.539829 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69213bf-4c74-4f29-9192-5391518f5b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:57.540124 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69213bf-4c74-4f29-9192-5391518f5b4c admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:57.540834 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69213bf-4c74-4f29-9192-5391518f5b4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:57.542906 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69213bf-4c74-4f29-9192-5391518f5b4c admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:57.545313 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:42:57.567078 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 783/1582] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:42:57 2022] POST /identity/v3/users => generated 279 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:42:57.567078 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69213bf-4c74-4f29-9192-5391518f5b4c admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:57.567394 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac51f25-f0ca-4766-8090-25bd8d266147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:57.567394 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac51f25-f0ca-4766-8090-25bd8d266147 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:57.567394 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac51f25-f0ca-4766-8090-25bd8d266147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:57.567394 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac51f25-f0ca-4766-8090-25bd8d266147 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:57.567722 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac51f25-f0ca-4766-8090-25bd8d266147 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:57.569030 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac51f25-f0ca-4766-8090-25bd8d266147 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:57.575220 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69213bf-4c74-4f29-9192-5391518f5b4c admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:57.581848 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c0539a-cc42-4cf6-8c81-e3029496df1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:57.583147 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c0539a-cc42-4cf6-8c81-e3029496df1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:57.583480 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c0539a-cc42-4cf6-8c81-e3029496df1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:57.583854 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c0539a-cc42-4cf6-8c81-e3029496df1c admin admin] PATH_INFO: `/v3/projects/16022d299eda48e6bf4675055870a140/users/26bcad09cb7749d780d151d3d2862795/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:57.592555 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 800/1583] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:42:57 2022] POST /identity/v3/users => generated 281 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:42:57.593429 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac51f25-f0ca-4766-8090-25bd8d266147 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:57.603780 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac51f25-f0ca-4766-8090-25bd8d266147 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:57.604333 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c0539a-cc42-4cf6-8c81-e3029496df1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=16022d299eda48e6bf4675055870a140, user_id=26bcad09cb7749d780d151d3d2862795, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:57.607332 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c0539a-cc42-4cf6-8c81-e3029496df1c admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:57.608501 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3357daf3-f097-4a6a-9403-219831a6e63d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:57.609450 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3357daf3-f097-4a6a-9403-219831a6e63d admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:57.609739 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3357daf3-f097-4a6a-9403-219831a6e63d admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:57.610045 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3357daf3-f097-4a6a-9403-219831a6e63d admin admin] PATH_INFO: `/v3/projects/3fbe05607bf945bbb756e7ed3764cccc/users/6e88db1bf9294decaf171756772a5066/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:57.624020 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 784/1584] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:42:57 2022] PUT /identity/v3/projects/16022d299eda48e6bf4675055870a140/users/26bcad09cb7749d780d151d3d2862795/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:42:57.625332 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c0539a-cc42-4cf6-8c81-e3029496df1c admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:57.628194 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3357daf3-f097-4a6a-9403-219831a6e63d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fbe05607bf945bbb756e7ed3764cccc, user_id=6e88db1bf9294decaf171756772a5066, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:57.629309 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: WARNING py.warnings [None req-3357daf3-f097-4a6a-9403-219831a6e63d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 16:42:57.629309 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: warnings.warn(msg) May 19 16:42:57.629309 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: May 19 16:42:57.631313 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3357daf3-f097-4a6a-9403-219831a6e63d admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:57.647439 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 801/1585] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:42:57 2022] PUT /identity/v3/projects/3fbe05607bf945bbb756e7ed3764cccc/users/6e88db1bf9294decaf171756772a5066/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:42:57.648358 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3357daf3-f097-4a6a-9403-219831a6e63d admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:57.664997 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:42:57.699555 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c0539a-cc42-4cf6-8c81-e3029496df1c admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:57.715981 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3357daf3-f097-4a6a-9403-219831a6e63d admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:57.721041 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ba291e-3ccd-4d6e-87ac-f9232c112e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:57.721759 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba291e-3ccd-4d6e-87ac-f9232c112e9e admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:57.722047 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba291e-3ccd-4d6e-87ac-f9232c112e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:57.722332 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba291e-3ccd-4d6e-87ac-f9232c112e9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:57.723691 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ba291e-3ccd-4d6e-87ac-f9232c112e9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:57.725384 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ba291e-3ccd-4d6e-87ac-f9232c112e9e admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:57.736927 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 802/1586] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:42:57 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:42:57.737833 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ba291e-3ccd-4d6e-87ac-f9232c112e9e admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:57.748394 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ba291e-3ccd-4d6e-87ac-f9232c112e9e admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:57.754006 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c201187-36db-4cdc-a433-f09573d24573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:57.754724 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c201187-36db-4cdc-a433-f09573d24573 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:57.755042 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c201187-36db-4cdc-a433-f09573d24573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:57.755322 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c201187-36db-4cdc-a433-f09573d24573 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:57.756066 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c201187-36db-4cdc-a433-f09573d24573 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:57.758343 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c201187-36db-4cdc-a433-f09573d24573 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:57.784323 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 803/1587] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:42:57 2022] POST /identity/v3/users => generated 286 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:42:57.784323 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c201187-36db-4cdc-a433-f09573d24573 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:57.795471 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b84383-7978-4900-9ad6-d4478fab88cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:57.796482 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c201187-36db-4cdc-a433-f09573d24573 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:57.796820 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b84383-7978-4900-9ad6-d4478fab88cb admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:57.797429 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b84383-7978-4900-9ad6-d4478fab88cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:57.797941 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b84383-7978-4900-9ad6-d4478fab88cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:57.799813 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b84383-7978-4900-9ad6-d4478fab88cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:57.803239 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b84383-7978-4900-9ad6-d4478fab88cb admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:57.803539 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72b56db-d3e4-4188-90c5-1c95860fc727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:57.804256 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b56db-d3e4-4188-90c5-1c95860fc727 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:57.804544 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b56db-d3e4-4188-90c5-1c95860fc727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:57.804828 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b56db-d3e4-4188-90c5-1c95860fc727 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:57.805629 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72b56db-d3e4-4188-90c5-1c95860fc727 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:57.807871 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72b56db-d3e4-4188-90c5-1c95860fc727 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:57.820633 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 785/1588] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:42:57 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:42:57.821885 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b84383-7978-4900-9ad6-d4478fab88cb admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:57.839211 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 804/1589] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:42:57 2022] POST /identity/v3/users => generated 295 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:42:57.840393 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72b56db-d3e4-4188-90c5-1c95860fc727 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:57.843991 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b84383-7978-4900-9ad6-d4478fab88cb admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:57.849221 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785538bc-24bb-496c-87d8-6c91dbaad4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:57.849963 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785538bc-24bb-496c-87d8-6c91dbaad4fa admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:57.850259 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785538bc-24bb-496c-87d8-6c91dbaad4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:57.850536 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785538bc-24bb-496c-87d8-6c91dbaad4fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:57.851187 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72b56db-d3e4-4188-90c5-1c95860fc727 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:57.851377 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785538bc-24bb-496c-87d8-6c91dbaad4fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:57.853288 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785538bc-24bb-496c-87d8-6c91dbaad4fa admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:57.856320 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a3dbe8-f2d7-4277-8187-827d9aa4e8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:57.856997 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3dbe8-f2d7-4277-8187-827d9aa4e8b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:57.857319 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3dbe8-f2d7-4277-8187-827d9aa4e8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:57.857630 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3dbe8-f2d7-4277-8187-827d9aa4e8b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:57.858377 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a3dbe8-f2d7-4277-8187-827d9aa4e8b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:57.860035 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a3dbe8-f2d7-4277-8187-827d9aa4e8b7 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:57.865205 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 786/1590] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:42:57 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:42:57.867197 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8136771-2c6d-4b8a-9cd3-8f268d00f255 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:57.867197 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8136771-2c6d-4b8a-9cd3-8f268d00f255 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:57.867197 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8136771-2c6d-4b8a-9cd3-8f268d00f255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:57.873768 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 805/1591] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:42:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:42:57.875265 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea3b11-2b18-4949-967c-f279f59a630e None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:57.875560 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea3b11-2b18-4949-967c-f279f59a630e None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:57.875841 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea3b11-2b18-4949-967c-f279f59a630e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:57.891658 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-e8136771-2c6d-4b8a-9cd3-8f268d00f255 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:42:57.893208 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-e8136771-2c6d-4b8a-9cd3-8f268d00f255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:42:57.896348 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-87ea3b11-2b18-4949-967c-f279f59a630e None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:42:57.897637 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-87ea3b11-2b18-4949-967c-f279f59a630e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:42:57.906867 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 787/1592] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:42:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:42:57.906867 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8136771-2c6d-4b8a-9cd3-8f268d00f255 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:57.911299 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 806/1593] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:42:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:42:57.912855 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea3b11-2b18-4949-967c-f279f59a630e None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:57.918692 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8136771-2c6d-4b8a-9cd3-8f268d00f255 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:57.923523 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712bf068-6765-432d-bbf7-24516aeace29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:57.925057 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712bf068-6765-432d-bbf7-24516aeace29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:57.925321 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712bf068-6765-432d-bbf7-24516aeace29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:57.925785 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712bf068-6765-432d-bbf7-24516aeace29 admin admin] PATH_INFO: `/v3/projects/5a6eafe486fe4b5cadac660a921721c6/users/a7a919d815074ed1aba725a56e198b47/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:57.929947 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea3b11-2b18-4949-967c-f279f59a630e None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:57.936932 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee753fbe-33b3-4a0d-baf7-9ba64e88ba8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:57.938132 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee753fbe-33b3-4a0d-baf7-9ba64e88ba8a admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:57.938652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee753fbe-33b3-4a0d-baf7-9ba64e88ba8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:57.939149 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee753fbe-33b3-4a0d-baf7-9ba64e88ba8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:57.941185 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee753fbe-33b3-4a0d-baf7-9ba64e88ba8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:57.943933 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712bf068-6765-432d-bbf7-24516aeace29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a6eafe486fe4b5cadac660a921721c6, user_id=a7a919d815074ed1aba725a56e198b47, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:57.944514 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee753fbe-33b3-4a0d-baf7-9ba64e88ba8a admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:57.946750 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712bf068-6765-432d-bbf7-24516aeace29 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:57.962337 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 807/1594] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:42:57 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:42:57.962337 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee753fbe-33b3-4a0d-baf7-9ba64e88ba8a admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:57.967232 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 788/1595] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:42:57 2022] PUT /identity/v3/projects/5a6eafe486fe4b5cadac660a921721c6/users/a7a919d815074ed1aba725a56e198b47/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:42:57.969286 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712bf068-6765-432d-bbf7-24516aeace29 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:57.982495 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee753fbe-33b3-4a0d-baf7-9ba64e88ba8a admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:58.036880 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:42:58.054177 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712bf068-6765-432d-bbf7-24516aeace29 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:58.060268 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb6aa3b-531c-4ffd-ae7f-fb462c920237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:58.061058 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb6aa3b-531c-4ffd-ae7f-fb462c920237 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:58.061290 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6b75e-a416-4e9d-afb5-ccc772941258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:58.061549 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb6aa3b-531c-4ffd-ae7f-fb462c920237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:58.061970 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb6aa3b-531c-4ffd-ae7f-fb462c920237 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:58.062209 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6b75e-a416-4e9d-afb5-ccc772941258 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:58.062499 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6b75e-a416-4e9d-afb5-ccc772941258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:58.062784 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6b75e-a416-4e9d-afb5-ccc772941258 admin admin] PATH_INFO: `/v3/projects/3e200a1b99664fc8b41757e2ae404f50/users/0695c868a2d844089fcda29ea3653d5c/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:58.063109 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb6aa3b-531c-4ffd-ae7f-fb462c920237 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:58.065812 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb6aa3b-531c-4ffd-ae7f-fb462c920237 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:58.078334 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 789/1596] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:42:57 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:42:58.079323 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb6aa3b-531c-4ffd-ae7f-fb462c920237 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:58.080747 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc6b75e-a416-4e9d-afb5-ccc772941258 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e200a1b99664fc8b41757e2ae404f50, user_id=0695c868a2d844089fcda29ea3653d5c, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:58.083723 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc6b75e-a416-4e9d-afb5-ccc772941258 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:58.090219 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb6aa3b-531c-4ffd-ae7f-fb462c920237 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:58.096100 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4895147c-68e7-4d93-af97-7a5f1981325f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:58.096789 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895147c-68e7-4d93-af97-7a5f1981325f admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:58.097070 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895147c-68e7-4d93-af97-7a5f1981325f admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:58.097380 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895147c-68e7-4d93-af97-7a5f1981325f admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:58.098118 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4895147c-68e7-4d93-af97-7a5f1981325f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:58.099827 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4895147c-68e7-4d93-af97-7a5f1981325f admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:58.101087 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 808/1597] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:42:57 2022] PUT /identity/v3/projects/3e200a1b99664fc8b41757e2ae404f50/users/0695c868a2d844089fcda29ea3653d5c/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:42:58.102539 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6b75e-a416-4e9d-afb5-ccc772941258 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:58.124951 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 790/1598] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:42:58 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:42:58.126546 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4895147c-68e7-4d93-af97-7a5f1981325f admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:58.148705 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:42:58.169998 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6b75e-a416-4e9d-afb5-ccc772941258 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:58.175421 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3cd9e-228a-4ab4-bb45-637dfb75c81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:58.176110 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3cd9e-228a-4ab4-bb45-637dfb75c81a admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:58.176586 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3cd9e-228a-4ab4-bb45-637dfb75c81a admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:58.176707 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3cd9e-228a-4ab4-bb45-637dfb75c81a admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:58.177444 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa3cd9e-228a-4ab4-bb45-637dfb75c81a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:58.179092 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa3cd9e-228a-4ab4-bb45-637dfb75c81a admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:58.189978 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 809/1599] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:42:58 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:42:58.190995 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3cd9e-228a-4ab4-bb45-637dfb75c81a admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:58.196003 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4895147c-68e7-4d93-af97-7a5f1981325f admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:58.247305 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a1d73d-2c8d-4a41-b2c7-993fa72be475 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:58.247979 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1d73d-2c8d-4a41-b2c7-993fa72be475 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:58.248275 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1d73d-2c8d-4a41-b2c7-993fa72be475 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:58.248561 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1d73d-2c8d-4a41-b2c7-993fa72be475 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:58.255777 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3cd9e-228a-4ab4-bb45-637dfb75c81a admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:58.258830 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a1d73d-2c8d-4a41-b2c7-993fa72be475 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:58.259808 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e944085-9d8b-4582-9d91-8f631a9ff86c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:58.260419 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a1d73d-2c8d-4a41-b2c7-993fa72be475 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:58.260590 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e944085-9d8b-4582-9d91-8f631a9ff86c service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:58.260743 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e944085-9d8b-4582-9d91-8f631a9ff86c service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:58.261026 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e944085-9d8b-4582-9d91-8f631a9ff86c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:58.270439 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 791/1600] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:42:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:42:58.270947 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e944085-9d8b-4582-9d91-8f631a9ff86c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:58.271260 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a1d73d-2c8d-4a41-b2c7-993fa72be475 service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:58.272426 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e944085-9d8b-4582-9d91-8f631a9ff86c service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:58.280800 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a1d73d-2c8d-4a41-b2c7-993fa72be475 service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:58.285235 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 810/1601] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:42:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:42:58.286822 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eeddad-4115-4470-8ce8-0ff31fd30622 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:58.287319 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eeddad-4115-4470-8ce8-0ff31fd30622 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:58.287441 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10403e-937d-4ec2-85c9-f3fdf02b43a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:58.287928 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eeddad-4115-4470-8ce8-0ff31fd30622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:58.288040 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10403e-937d-4ec2-85c9-f3fdf02b43a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:58.288279 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10403e-937d-4ec2-85c9-f3fdf02b43a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:58.288562 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10403e-937d-4ec2-85c9-f3fdf02b43a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:58.289338 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b10403e-937d-4ec2-85c9-f3fdf02b43a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:58.290964 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b10403e-937d-4ec2-85c9-f3fdf02b43a2 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:58.303994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 792/1602] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:42:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:42:58.305106 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10403e-937d-4ec2-85c9-f3fdf02b43a2 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:58.316926 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10403e-937d-4ec2-85c9-f3fdf02b43a2 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:58.322208 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e037fa3-af5f-45c3-a226-c694c5fae088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:58.322971 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e037fa3-af5f-45c3-a226-c694c5fae088 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:58.323612 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e037fa3-af5f-45c3-a226-c694c5fae088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:58.323612 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e037fa3-af5f-45c3-a226-c694c5fae088 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:58.324317 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e037fa3-af5f-45c3-a226-c694c5fae088 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:58.326034 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e037fa3-af5f-45c3-a226-c694c5fae088 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:58.329630 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-c7eeddad-4115-4470-8ce8-0ff31fd30622 None None] MFA Rules not processed for user `a7a919d815074ed1aba725a56e198b47`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:42:58.330943 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-c7eeddad-4115-4470-8ce8-0ff31fd30622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:42:58.337308 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 793/1603] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:42:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:42:58.342453 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ac4d0-3571-4426-be16-8b33e2b0d2b9 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:58.342453 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ac4d0-3571-4426-be16-8b33e2b0d2b9 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:58.342690 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ac4d0-3571-4426-be16-8b33e2b0d2b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:58.362459 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-743ac4d0-3571-4426-be16-8b33e2b0d2b9 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:42:58.363697 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-743ac4d0-3571-4426-be16-8b33e2b0d2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:42:58.373538 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 794/1604] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:42:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:42:58.375179 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743ac4d0-3571-4426-be16-8b33e2b0d2b9 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:58.385380 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743ac4d0-3571-4426-be16-8b33e2b0d2b9 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:58.391028 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:42:58.391661 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9905e6ec-37cb-48e3-bb7e-470327d8d40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:58.393349 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9905e6ec-37cb-48e3-bb7e-470327d8d40a admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:58.393895 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9905e6ec-37cb-48e3-bb7e-470327d8d40a admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:58.394174 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9905e6ec-37cb-48e3-bb7e-470327d8d40a admin admin] PATH_INFO: `/v3/projects/a5a2b3a6de8f4fdc8c7573a3fd3b4bf7/users/00680c852f4a4592bc1c7a5e9cbcbdec/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:58.412486 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9905e6ec-37cb-48e3-bb7e-470327d8d40a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5a2b3a6de8f4fdc8c7573a3fd3b4bf7, user_id=00680c852f4a4592bc1c7a5e9cbcbdec, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:58.415310 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9905e6ec-37cb-48e3-bb7e-470327d8d40a admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:58.420791 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 811/1605] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:42:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:42:58.420791 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eeddad-4115-4470-8ce8-0ff31fd30622 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:58.434502 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eeddad-4115-4470-8ce8-0ff31fd30622 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:58.447791 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 795/1606] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:42:58 2022] PUT /identity/v3/projects/a5a2b3a6de8f4fdc8c7573a3fd3b4bf7/users/00680c852f4a4592bc1c7a5e9cbcbdec/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:42:58.448838 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9905e6ec-37cb-48e3-bb7e-470327d8d40a admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:58.531707 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5adb52-313b-403e-b126-2121941c706c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:58.532986 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5adb52-313b-403e-b126-2121941c706c admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:58.534146 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5adb52-313b-403e-b126-2121941c706c admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:58.534640 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5adb52-313b-403e-b126-2121941c706c admin admin] PATH_INFO: `/v3/projects/87e169cd9e5e4cc3b391cdb01d11520d/users/df3b4f232bd9411999e26d6adc77837f/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:58.552913 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5adb52-313b-403e-b126-2121941c706c admin admin] RBAC: Authorizing `identity:create_grant(project_id=87e169cd9e5e4cc3b391cdb01d11520d, user_id=df3b4f232bd9411999e26d6adc77837f, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:58.555023 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9905e6ec-37cb-48e3-bb7e-470327d8d40a admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:58.555758 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5adb52-313b-403e-b126-2121941c706c admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:58.562809 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1b1467-1079-4d98-a924-48a7e9281784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:58.563744 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b1467-1079-4d98-a924-48a7e9281784 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:58.564030 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b1467-1079-4d98-a924-48a7e9281784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:58.564463 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b1467-1079-4d98-a924-48a7e9281784 admin admin] PATH_INFO: `/v3/projects/e0ed7ea223d744b283eacf61514bc9aa/users/cd00835e169a4c42bea59e8204e7c09e/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:58.573716 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 812/1607] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:42:58 2022] PUT /identity/v3/projects/87e169cd9e5e4cc3b391cdb01d11520d/users/df3b4f232bd9411999e26d6adc77837f/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:42:58.574951 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5adb52-313b-403e-b126-2121941c706c admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:58.602826 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1b1467-1079-4d98-a924-48a7e9281784 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0ed7ea223d744b283eacf61514bc9aa, user_id=cd00835e169a4c42bea59e8204e7c09e, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:58.607590 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1b1467-1079-4d98-a924-48a7e9281784 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:58.628259 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 796/1608] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:42:58 2022] PUT /identity/v3/projects/e0ed7ea223d744b283eacf61514bc9aa/users/cd00835e169a4c42bea59e8204e7c09e/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:42:58.630018 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1b1467-1079-4d98-a924-48a7e9281784 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:58.646728 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-cc1b1467-1079-4d98-a924-48a7e9281784 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:42:58.659449 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:42:58.689019 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5adb52-313b-403e-b126-2121941c706c admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:58.767171 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1b1467-1079-4d98-a924-48a7e9281784 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:58.772471 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e382e-5f31-4223-bdb3-79f5b676cbcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:58.773169 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e382e-5f31-4223-bdb3-79f5b676cbcd service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:58.773469 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e382e-5f31-4223-bdb3-79f5b676cbcd service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:58.773748 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e382e-5f31-4223-bdb3-79f5b676cbcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:58.776368 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e7f28a-b075-4613-85ce-c877ae1b2189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:58.777561 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7f28a-b075-4613-85ce-c877ae1b2189 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:58.777936 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7f28a-b075-4613-85ce-c877ae1b2189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:58.778225 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7f28a-b075-4613-85ce-c877ae1b2189 admin admin] PATH_INFO: `/v3/projects/bef5d1b475ef4228a77b87964b0b8ce1/users/86b78ffb46894509bf8e81a154073885/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:58.787978 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27e382e-5f31-4223-bdb3-79f5b676cbcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:58.789710 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27e382e-5f31-4223-bdb3-79f5b676cbcd service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:58.799029 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e7f28a-b075-4613-85ce-c877ae1b2189 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bef5d1b475ef4228a77b87964b0b8ce1, user_id=86b78ffb46894509bf8e81a154073885, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:58.801843 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e7f28a-b075-4613-85ce-c877ae1b2189 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:58.808626 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 797/1609] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:42:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:42:58.811146 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01191ee5-97bd-4ab7-baf5-aecff35bfeab None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:58.811475 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01191ee5-97bd-4ab7-baf5-aecff35bfeab None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:58.811873 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01191ee5-97bd-4ab7-baf5-aecff35bfeab None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:58.814131 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 798/1610] 10.222.0.36 () {54 vars in 961 bytes} [Thu May 19 16:42:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:42:58.815231 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c827f-522c-4e8b-9ec8-aeb0865a8de6 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:58.815669 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c827f-522c-4e8b-9ec8-aeb0865a8de6 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:58.816113 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c827f-522c-4e8b-9ec8-aeb0865a8de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:58.839604 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 813/1611] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:42:58 2022] PUT /identity/v3/projects/bef5d1b475ef4228a77b87964b0b8ce1/users/86b78ffb46894509bf8e81a154073885/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:42:58.842967 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee15c57-2eac-4d61-9fa0-f1c085ca4772 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:58.845666 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee15c57-2eac-4d61-9fa0-f1c085ca4772 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:58.846183 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee15c57-2eac-4d61-9fa0-f1c085ca4772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:58.856624 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:42:58.868473 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-534c827f-522c-4e8b-9ec8-aeb0865a8de6 None None] MFA Rules not processed for user `00680c852f4a4592bc1c7a5e9cbcbdec`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:42:58.869853 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-534c827f-522c-4e8b-9ec8-aeb0865a8de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:42:58.922947 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-bee15c57-2eac-4d61-9fa0-f1c085ca4772 None None] MFA Rules not processed for user `df3b4f232bd9411999e26d6adc77837f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:42:58.926280 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-bee15c57-2eac-4d61-9fa0-f1c085ca4772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:42:59.009416 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 799/1612] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:42:58 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:42:59.011374 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a664fbd-cfbb-42e8-abba-adb804ad08b6 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:59.011740 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a664fbd-cfbb-42e8-abba-adb804ad08b6 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:59.012095 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a664fbd-cfbb-42e8-abba-adb804ad08b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:59.054802 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 814/1613] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:42:58 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:42:59.056350 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a83d6-d876-476a-a135-4e67b4a0e956 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:59.056711 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a83d6-d876-476a-a135-4e67b4a0e956 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:59.057067 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a83d6-d876-476a-a135-4e67b4a0e956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:59.061023 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-2a664fbd-cfbb-42e8-abba-adb804ad08b6 None None] MFA Rules not processed for user `cd00835e169a4c42bea59e8204e7c09e`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:42:59.065371 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-2a664fbd-cfbb-42e8-abba-adb804ad08b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:42:59.085006 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-1c3a83d6-d876-476a-a135-4e67b4a0e956 None None] MFA Rules not processed for user `58bdeada00b74b7b8e186e6dc51e6dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:42:59.086907 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-1c3a83d6-d876-476a-a135-4e67b4a0e956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:42:59.190012 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 800/1614] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:42:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:42:59.190012 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9abef76-73ac-484a-91fc-252de75696a5 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:59.190012 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9abef76-73ac-484a-91fc-252de75696a5 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:59.190012 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9abef76-73ac-484a-91fc-252de75696a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:59.255924 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-b9abef76-73ac-484a-91fc-252de75696a5 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:42:59.262287 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-b9abef76-73ac-484a-91fc-252de75696a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:42:59.263612 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 815/1615] 10.222.0.36 () {58 vars in 1058 bytes} [Thu May 19 16:42:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:42:59.264572 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3a83d6-d876-476a-a135-4e67b4a0e956 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:59.321716 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:42:59.388386 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3a83d6-d876-476a-a135-4e67b4a0e956 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:59.394507 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc23744-0306-4a32-8645-27cd4fd6cdf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:59.395343 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 801/1616] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:42:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:42:59.396445 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9abef76-73ac-484a-91fc-252de75696a5 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:59.399007 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc23744-0306-4a32-8645-27cd4fd6cdf0 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:59.399343 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc23744-0306-4a32-8645-27cd4fd6cdf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:59.400242 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc23744-0306-4a32-8645-27cd4fd6cdf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:59.400786 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc23744-0306-4a32-8645-27cd4fd6cdf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:59.403623 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc23744-0306-4a32-8645-27cd4fd6cdf0 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:59.434701 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9abef76-73ac-484a-91fc-252de75696a5 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:59.455324 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96abeb2-b23f-445d-b146-f52f27d7243b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:59.455655 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 816/1617] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:42:59 2022] POST /identity/v3/projects => generated 357 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:42:59.456037 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96abeb2-b23f-445d-b146-f52f27d7243b admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:59.456583 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96abeb2-b23f-445d-b146-f52f27d7243b admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:59.457125 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96abeb2-b23f-445d-b146-f52f27d7243b admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:59.457692 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3aec0-9799-4af8-883e-a87aa85ccc6a None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:59.458100 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3aec0-9799-4af8-883e-a87aa85ccc6a None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:59.458547 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3aec0-9799-4af8-883e-a87aa85ccc6a None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:59.459474 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96abeb2-b23f-445d-b146-f52f27d7243b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:59.460931 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 817/1618] 10.222.0.36 () {54 vars in 961 bytes} [Thu May 19 16:42:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:42:59.461935 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb3aec0-9799-4af8-883e-a87aa85ccc6a None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:59.463820 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96abeb2-b23f-445d-b146-f52f27d7243b admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:59.482548 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:42:59.526758 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 802/1619] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:42:59 2022] POST /identity/v3/projects => generated 363 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:42:59.527587 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96abeb2-b23f-445d-b146-f52f27d7243b admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:59.594506 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb3aec0-9799-4af8-883e-a87aa85ccc6a None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:59.616287 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96abeb2-b23f-445d-b146-f52f27d7243b admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:59.628262 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88534c8-42bd-4c3e-9359-504d0b4888d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:59.629784 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88534c8-42bd-4c3e-9359-504d0b4888d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:59.630690 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88534c8-42bd-4c3e-9359-504d0b4888d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:59.632653 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88534c8-42bd-4c3e-9359-504d0b4888d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:59.701259 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28444aca-1317-48fe-9590-0c1b8273b918 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:59.701773 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28444aca-1317-48fe-9590-0c1b8273b918 service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:59.702481 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28444aca-1317-48fe-9590-0c1b8273b918 service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:59.702481 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28444aca-1317-48fe-9590-0c1b8273b918 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:59.808797 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28444aca-1317-48fe-9590-0c1b8273b918 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:59.811646 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28444aca-1317-48fe-9590-0c1b8273b918 service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:59.820630 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88534c8-42bd-4c3e-9359-504d0b4888d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:59.825765 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88534c8-42bd-4c3e-9359-504d0b4888d4 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:59.850890 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 803/1620] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:42:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:42:59.852178 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88534c8-42bd-4c3e-9359-504d0b4888d4 service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:59.853292 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 818/1621] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:42:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:42:59.854830 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ca9f8-3cef-41fc-8618-3e4c60994f67 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:59.855127 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ca9f8-3cef-41fc-8618-3e4c60994f67 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:59.855408 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ca9f8-3cef-41fc-8618-3e4c60994f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:59.882927 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88534c8-42bd-4c3e-9359-504d0b4888d4 service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:59.903162 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2637ddf0-bcd5-4379-a104-ec16fe14b591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:59.903934 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637ddf0-bcd5-4379-a104-ec16fe14b591 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:59.904313 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637ddf0-bcd5-4379-a104-ec16fe14b591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:59.904744 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637ddf0-bcd5-4379-a104-ec16fe14b591 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:59.905736 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2637ddf0-bcd5-4379-a104-ec16fe14b591 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:59.908200 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2637ddf0-bcd5-4379-a104-ec16fe14b591 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:59.910803 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-b98ca9f8-3cef-41fc-8618-3e4c60994f67 None None] MFA Rules not processed for user `58bdeada00b74b7b8e186e6dc51e6dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:42:59.912098 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b98ca9f8-3cef-41fc-8618-3e4c60994f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:42:59.936051 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 819/1622] 10.222.0.36 () {58 vars in 1058 bytes} [Thu May 19 16:42:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:42:59.938305 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ca9f8-3cef-41fc-8618-3e4c60994f67 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:42:59.950528 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 804/1623] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:42:59 2022] POST /identity/v3/users => generated 285 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:42:59.952603 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47997dc0-9f6a-4251-92ea-99a37033e734 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:59.955844 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47997dc0-9f6a-4251-92ea-99a37033e734 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:59.955844 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47997dc0-9f6a-4251-92ea-99a37033e734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:59.964303 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ca9f8-3cef-41fc-8618-3e4c60994f67 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:42:59.979780 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e0ecc-d215-48c4-a19e-e6fe328e6546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:42:59.980633 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e0ecc-d215-48c4-a19e-e6fe328e6546 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:42:59.981027 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e0ecc-d215-48c4-a19e-e6fe328e6546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:42:59.981384 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e0ecc-d215-48c4-a19e-e6fe328e6546 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:42:59.982110 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e0ecc-d215-48c4-a19e-e6fe328e6546 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:42:59.984495 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e0ecc-d215-48c4-a19e-e6fe328e6546 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:42:59.994841 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-47997dc0-9f6a-4251-92ea-99a37033e734 None None] MFA Rules not processed for user `26bcad09cb7749d780d151d3d2862795`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:42:59.996331 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-47997dc0-9f6a-4251-92ea-99a37033e734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:00.019835 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 820/1624] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:42:59 2022] POST /identity/v3/users => generated 288 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:43:00.020440 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e0ecc-d215-48c4-a19e-e6fe328e6546 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:00.050258 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e0ecc-d215-48c4-a19e-e6fe328e6546 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:00.052075 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:00.061212 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff5f96-4431-406f-8f5c-86c827e7c3d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:00.063485 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff5f96-4431-406f-8f5c-86c827e7c3d1 service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:00.064911 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff5f96-4431-406f-8f5c-86c827e7c3d1 service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:00.065628 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff5f96-4431-406f-8f5c-86c827e7c3d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:00.126774 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:00.139218 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 805/1625] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:42:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:00.140725 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47997dc0-9f6a-4251-92ea-99a37033e734 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:00.157500 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47997dc0-9f6a-4251-92ea-99a37033e734 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:00.164208 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ea68e7-7316-4954-b380-7130611a6272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:00.165279 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea68e7-7316-4954-b380-7130611a6272 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:00.165670 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea68e7-7316-4954-b380-7130611a6272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:00.166143 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea68e7-7316-4954-b380-7130611a6272 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:00.167287 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ea68e7-7316-4954-b380-7130611a6272 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:00.171859 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ea68e7-7316-4954-b380-7130611a6272 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:00.200952 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 806/1626] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:43:00.202358 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ea68e7-7316-4954-b380-7130611a6272 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:00.217606 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ea68e7-7316-4954-b380-7130611a6272 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:00.223498 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e702c2-fb54-4a14-90e0-0a9ced8aa1cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:00.224395 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e702c2-fb54-4a14-90e0-0a9ced8aa1cb admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:00.224829 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e702c2-fb54-4a14-90e0-0a9ced8aa1cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:00.225458 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e702c2-fb54-4a14-90e0-0a9ced8aa1cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:00.226501 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e702c2-fb54-4a14-90e0-0a9ced8aa1cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:00.229121 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e702c2-fb54-4a14-90e0-0a9ced8aa1cb admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:00.240881 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfff5f96-4431-406f-8f5c-86c827e7c3d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:00.244982 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 807/1627] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:43:00.246776 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4552bd-a558-4d9d-bb73-0c29a49a3057 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:00.247349 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4552bd-a558-4d9d-bb73-0c29a49a3057 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:00.247822 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4552bd-a558-4d9d-bb73-0c29a49a3057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:00.248607 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfff5f96-4431-406f-8f5c-86c827e7c3d1 service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:00.272700 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-cf4552bd-a558-4d9d-bb73-0c29a49a3057 None None] MFA Rules not processed for user `26bcad09cb7749d780d151d3d2862795`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:00.274867 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-cf4552bd-a558-4d9d-bb73-0c29a49a3057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:00.284130 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 808/1628] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:00.285248 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4552bd-a558-4d9d-bb73-0c29a49a3057 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:00.298121 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4552bd-a558-4d9d-bb73-0c29a49a3057 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:00.303560 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 821/1629] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:00 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:43:00.304466 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff5f96-4431-406f-8f5c-86c827e7c3d1 service nova] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:00.306554 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16193f3-5a63-4496-baef-f75a434a0f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:00.308850 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16193f3-5a63-4496-baef-f75a434a0f1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:00.309713 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16193f3-5a63-4496-baef-f75a434a0f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:00.310653 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16193f3-5a63-4496-baef-f75a434a0f1d admin admin] PATH_INFO: `/v3/projects/776ce9b37bfb44909f448d6eea08599a/users/30409d12ce2040cd9a76736e8569007b/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:00.335316 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16193f3-5a63-4496-baef-f75a434a0f1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=776ce9b37bfb44909f448d6eea08599a, user_id=30409d12ce2040cd9a76736e8569007b, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:00.338304 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16193f3-5a63-4496-baef-f75a434a0f1d admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:00.356312 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff5f96-4431-406f-8f5c-86c827e7c3d1 service nova] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:00.357928 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 809/1630] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:43:00 2022] PUT /identity/v3/projects/776ce9b37bfb44909f448d6eea08599a/users/30409d12ce2040cd9a76736e8569007b/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:00.359548 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7486f051-41be-4193-b4c7-3a3c4ebb98f7 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:00.359839 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7486f051-41be-4193-b4c7-3a3c4ebb98f7 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:00.360160 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7486f051-41be-4193-b4c7-3a3c4ebb98f7 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:00.362594 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 810/1631] 10.222.0.36 () {54 vars in 912 bytes} [Thu May 19 16:43:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:43:00.363395 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7486f051-41be-4193-b4c7-3a3c4ebb98f7 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:00.496996 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7486f051-41be-4193-b4c7-3a3c4ebb98f7 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:00.512721 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018aebfd-0334-407a-86be-8946773425e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:00.514355 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018aebfd-0334-407a-86be-8946773425e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:00.514649 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018aebfd-0334-407a-86be-8946773425e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:00.514935 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018aebfd-0334-407a-86be-8946773425e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:00.518755 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018aebfd-0334-407a-86be-8946773425e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:00.522973 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018aebfd-0334-407a-86be-8946773425e5 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:00.556788 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 811/1632] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:43:00.557765 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66d838b-d6f9-4e40-b9a9-4abc6c3db174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:00.558812 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d838b-d6f9-4e40-b9a9-4abc6c3db174 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:00.559755 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b66aa-5987-423e-9e42-b3d04f01f8e9 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:00.560055 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b66aa-5987-423e-9e42-b3d04f01f8e9 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:00.560339 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b66aa-5987-423e-9e42-b3d04f01f8e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:00.561494 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d838b-d6f9-4e40-b9a9-4abc6c3db174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:00.561494 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d838b-d6f9-4e40-b9a9-4abc6c3db174 admin admin] PATH_INFO: `/v3/projects/384a0628899d4716a7a1b0a7a54523be/users/219fc44a279147068b3aa624dd832b1e/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:00.600169 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66d838b-d6f9-4e40-b9a9-4abc6c3db174 admin admin] RBAC: Authorizing `identity:create_grant(project_id=384a0628899d4716a7a1b0a7a54523be, user_id=219fc44a279147068b3aa624dd832b1e, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:00.604941 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66d838b-d6f9-4e40-b9a9-4abc6c3db174 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:00.606454 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-696b66aa-5987-423e-9e42-b3d04f01f8e9 None None] MFA Rules not processed for user `58bdeada00b74b7b8e186e6dc51e6dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:00.613595 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-696b66aa-5987-423e-9e42-b3d04f01f8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:00.639839 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 822/1633] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:43:00 2022] PUT /identity/v3/projects/384a0628899d4716a7a1b0a7a54523be/users/219fc44a279147068b3aa624dd832b1e/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:00.643026 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66d838b-d6f9-4e40-b9a9-4abc6c3db174 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:00.814144 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:00.843961 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 812/1634] 10.222.0.36 () {58 vars in 1009 bytes} [Thu May 19 16:43:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:00.844888 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b66aa-5987-423e-9e42-b3d04f01f8e9 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:00.863182 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66d838b-d6f9-4e40-b9a9-4abc6c3db174 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:00.872608 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b66aa-5987-423e-9e42-b3d04f01f8e9 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:00.875625 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab26ef9-0a5f-4787-9aa1-d64fae8c4a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:00.876757 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab26ef9-0a5f-4787-9aa1-d64fae8c4a5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:00.877525 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c7215b-7cc0-4b4d-ac62-800dca490ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:00.877946 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab26ef9-0a5f-4787-9aa1-d64fae8c4a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:00.878587 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7215b-7cc0-4b4d-ac62-800dca490ebc admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:00.878719 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab26ef9-0a5f-4787-9aa1-d64fae8c4a5b admin admin] PATH_INFO: `/v3/projects/776ce9b37bfb44909f448d6eea08599a/users/30409d12ce2040cd9a76736e8569007b/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:00.878806 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7215b-7cc0-4b4d-ac62-800dca490ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:00.879130 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7215b-7cc0-4b4d-ac62-800dca490ebc admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:00.879944 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c7215b-7cc0-4b4d-ac62-800dca490ebc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:00.881693 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c7215b-7cc0-4b4d-ac62-800dca490ebc admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:00.881831 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab26ef9-0a5f-4787-9aa1-d64fae8c4a5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=776ce9b37bfb44909f448d6eea08599a, user_id=30409d12ce2040cd9a76736e8569007b, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:00.893494 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab26ef9-0a5f-4787-9aa1-d64fae8c4a5b admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:00.902785 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 813/1635] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:43:00.903582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c7215b-7cc0-4b4d-ac62-800dca490ebc admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:00.921679 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 823/1636] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:43:00 2022] PUT /identity/v3/projects/776ce9b37bfb44909f448d6eea08599a/users/30409d12ce2040cd9a76736e8569007b/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:00.923567 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e352934-c513-4434-892d-05389044b246 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:00.923933 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e352934-c513-4434-892d-05389044b246 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:00.924229 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e352934-c513-4434-892d-05389044b246 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:00.925955 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 824/1637] 10.222.0.36 () {54 vars in 912 bytes} [Thu May 19 16:43:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:43:00.926685 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e352934-c513-4434-892d-05389044b246 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:01.051669 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c7215b-7cc0-4b4d-ac62-800dca490ebc admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:01.070147 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:01.162059 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e352934-c513-4434-892d-05389044b246 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:01.180897 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859ffc26-7c98-4610-84ff-9ba018c9623d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:01.183491 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ffc26-7c98-4610-84ff-9ba018c9623d admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:01.189872 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ffc26-7c98-4610-84ff-9ba018c9623d admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:01.190222 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ffc26-7c98-4610-84ff-9ba018c9623d admin admin] PATH_INFO: `/v3/projects/384a0628899d4716a7a1b0a7a54523be/users/219fc44a279147068b3aa624dd832b1e/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:01.191319 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde20f99-0c90-4d84-9770-b1ca5337b6de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:01.192248 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde20f99-0c90-4d84-9770-b1ca5337b6de service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:01.192799 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde20f99-0c90-4d84-9770-b1ca5337b6de service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:01.193935 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde20f99-0c90-4d84-9770-b1ca5337b6de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:01.199357 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859ffc26-7c98-4610-84ff-9ba018c9623d admin admin] RBAC: Authorizing `identity:create_grant(project_id=384a0628899d4716a7a1b0a7a54523be, user_id=219fc44a279147068b3aa624dd832b1e, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:01.210882 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859ffc26-7c98-4610-84ff-9ba018c9623d admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:01.249176 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 825/1638] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:43:00 2022] PUT /identity/v3/projects/384a0628899d4716a7a1b0a7a54523be/users/219fc44a279147068b3aa624dd832b1e/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:01.255716 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c421b42-f88a-49e7-a250-7e2fd5480351 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:01.255925 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c421b42-f88a-49e7-a250-7e2fd5480351 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:01.256136 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c421b42-f88a-49e7-a250-7e2fd5480351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:01.341993 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-1c421b42-f88a-49e7-a250-7e2fd5480351 None None] MFA Rules not processed for user `30409d12ce2040cd9a76736e8569007b`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:01.344702 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-1c421b42-f88a-49e7-a250-7e2fd5480351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:01.360842 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde20f99-0c90-4d84-9770-b1ca5337b6de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:01.362686 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde20f99-0c90-4d84-9770-b1ca5337b6de service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:01.407763 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 814/1639] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:43:01.420509 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197ea8b6-2550-4632-9549-8f8bf431ebed None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:01.420938 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197ea8b6-2550-4632-9549-8f8bf431ebed None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:01.421353 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197ea8b6-2550-4632-9549-8f8bf431ebed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:01.462175 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 826/1640] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:01 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:01.463735 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ba5c2-a038-47d4-a464-2371b674a5bb None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:01.464033 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ba5c2-a038-47d4-a464-2371b674a5bb None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:01.464609 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ba5c2-a038-47d4-a464-2371b674a5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:01.479963 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-197ea8b6-2550-4632-9549-8f8bf431ebed None None] MFA Rules not processed for user `58bdeada00b74b7b8e186e6dc51e6dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:01.483174 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-197ea8b6-2550-4632-9549-8f8bf431ebed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:01.504866 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-1d4ba5c2-a038-47d4-a464-2371b674a5bb None None] MFA Rules not processed for user `219fc44a279147068b3aa624dd832b1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:01.506328 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-1d4ba5c2-a038-47d4-a464-2371b674a5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:01.624866 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:01.676761 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 827/1641] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:01 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:01.678444 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ba5c2-a038-47d4-a464-2371b674a5bb None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:01.748273 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 815/1642] 10.222.0.36 () {58 vars in 1009 bytes} [Thu May 19 16:43:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:01.788715 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:01.808486 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ba5c2-a038-47d4-a464-2371b674a5bb None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:01.814099 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2611dc0-25b1-49ed-90ee-6b8ccc3addda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:01.815691 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2611dc0-25b1-49ed-90ee-6b8ccc3addda service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:01.816252 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2611dc0-25b1-49ed-90ee-6b8ccc3addda service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:01.816856 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2611dc0-25b1-49ed-90ee-6b8ccc3addda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:01.848030 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2611dc0-25b1-49ed-90ee-6b8ccc3addda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:01.852745 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2611dc0-25b1-49ed-90ee-6b8ccc3addda service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:01.903539 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 828/1643] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:01 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:43:02.008812 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197ea8b6-2550-4632-9549-8f8bf431ebed None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:02.022141 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197ea8b6-2550-4632-9549-8f8bf431ebed None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:02.029871 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0683bb11-938d-4fb5-9347-98c5e48123dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:02.031343 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0683bb11-938d-4fb5-9347-98c5e48123dc service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:02.031637 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0683bb11-938d-4fb5-9347-98c5e48123dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:02.031921 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0683bb11-938d-4fb5-9347-98c5e48123dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:02.057964 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0683bb11-938d-4fb5-9347-98c5e48123dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:02.067920 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0683bb11-938d-4fb5-9347-98c5e48123dc service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:02.091189 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 816/1644] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:02 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:43:02.198632 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2611dc0-25b1-49ed-90ee-6b8ccc3addda service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:02.208964 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2611dc0-25b1-49ed-90ee-6b8ccc3addda service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:02.224392 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f0f0c3-cd0e-4dcd-9172-c9ab8948d668 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:02.225074 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f0f0c3-cd0e-4dcd-9172-c9ab8948d668 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:02.225392 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f0f0c3-cd0e-4dcd-9172-c9ab8948d668 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:02.225677 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f0f0c3-cd0e-4dcd-9172-c9ab8948d668 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:02.254367 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f0f0c3-cd0e-4dcd-9172-c9ab8948d668 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:02.257859 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f0f0c3-cd0e-4dcd-9172-c9ab8948d668 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:02.295767 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 829/1645] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:43:03.037281 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0683bb11-938d-4fb5-9347-98c5e48123dc service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:03.048387 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f0f0c3-cd0e-4dcd-9172-c9ab8948d668 service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:03.061007 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f0f0c3-cd0e-4dcd-9172-c9ab8948d668 service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:03.063959 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0683bb11-938d-4fb5-9347-98c5e48123dc service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:03.067550 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a521f7-8365-408d-b0ec-e1174b16e41a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:03.068479 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a521f7-8365-408d-b0ec-e1174b16e41a service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:03.068771 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a521f7-8365-408d-b0ec-e1174b16e41a service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:03.070355 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a521f7-8365-408d-b0ec-e1174b16e41a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:03.083719 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e520cb-2570-41dd-80fd-88a98fa4dba5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:03.084417 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e520cb-2570-41dd-80fd-88a98fa4dba5 service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:03.084697 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e520cb-2570-41dd-80fd-88a98fa4dba5 service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:03.084981 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e520cb-2570-41dd-80fd-88a98fa4dba5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:03.270150 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a521f7-8365-408d-b0ec-e1174b16e41a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:03.271661 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a521f7-8365-408d-b0ec-e1174b16e41a service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:03.280785 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e520cb-2570-41dd-80fd-88a98fa4dba5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:03.287099 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e520cb-2570-41dd-80fd-88a98fa4dba5 service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:03.303780 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 830/1646] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:03 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 16:43:03.304651 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a521f7-8365-408d-b0ec-e1174b16e41a service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:03.316386 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 817/1647] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:43:03.335205 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a521f7-8365-408d-b0ec-e1174b16e41a service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:03.340084 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d33f2a-67c2-4965-8e76-1d461101c896 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:03.340891 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d33f2a-67c2-4965-8e76-1d461101c896 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:03.341421 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d33f2a-67c2-4965-8e76-1d461101c896 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:03.341885 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d33f2a-67c2-4965-8e76-1d461101c896 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:03.545797 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d33f2a-67c2-4965-8e76-1d461101c896 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:03.547777 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d33f2a-67c2-4965-8e76-1d461101c896 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:03.576794 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 831/1648] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:03 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:43:05.003375 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d140d-0d56-4b62-8243-6930f5df02a9 None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:05.003831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d140d-0d56-4b62-8243-6930f5df02a9 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:05.003831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d140d-0d56-4b62-8243-6930f5df02a9 None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:05.006030 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 818/1649] 10.222.0.36 () {54 vars in 912 bytes} [Thu May 19 16:43:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:43:05.021294 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198733d3-944d-48a5-8512-78f92a4b1c54 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:05.022190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198733d3-944d-48a5-8512-78f92a4b1c54 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:05.024244 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198733d3-944d-48a5-8512-78f92a4b1c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:05.186816 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-198733d3-944d-48a5-8512-78f92a4b1c54 None None] MFA Rules not processed for user `33472643b65e4fe08fd5b3aa2c618133`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:05.188286 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-198733d3-944d-48a5-8512-78f92a4b1c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:05.367139 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 832/1650] 10.222.0.36 () {58 vars in 1009 bytes} [Thu May 19 16:43:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:05.434556 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066d140d-0d56-4b62-8243-6930f5df02a9 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:05.512123 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066d140d-0d56-4b62-8243-6930f5df02a9 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:05.525588 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d351126f-dd3f-4046-affa-b73511d650c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:05.526266 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351126f-dd3f-4046-affa-b73511d650c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:05.526555 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351126f-dd3f-4046-affa-b73511d650c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:05.526860 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351126f-dd3f-4046-affa-b73511d650c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:05.580304 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d351126f-dd3f-4046-affa-b73511d650c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:05.584398 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d351126f-dd3f-4046-affa-b73511d650c4 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:05.621939 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 819/1651] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:43:07.293638 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b3297-cffc-4e19-9b2f-ffc34dba2f8c None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:07.293638 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b3297-cffc-4e19-9b2f-ffc34dba2f8c None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:07.293638 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b3297-cffc-4e19-9b2f-ffc34dba2f8c None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:07.294708 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 833/1652] 10.222.0.36 () {54 vars in 960 bytes} [Thu May 19 16:43:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:43:07.300712 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa952c45-8f30-4353-a9c9-5613ac4c0ac5 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:07.302261 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa952c45-8f30-4353-a9c9-5613ac4c0ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:07.302764 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa952c45-8f30-4353-a9c9-5613ac4c0ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:07.361598 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-fa952c45-8f30-4353-a9c9-5613ac4c0ac5 None None] MFA Rules not processed for user `c6e158e5c87f4954aeb297f42dabdc16`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:07.369835 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-fa952c45-8f30-4353-a9c9-5613ac4c0ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:07.585228 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 820/1653] 10.222.0.36 () {58 vars in 1057 bytes} [Thu May 19 16:43:07 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:07.585770 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255b3297-cffc-4e19-9b2f-ffc34dba2f8c None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:07.596680 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:07.609609 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255b3297-cffc-4e19-9b2f-ffc34dba2f8c None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:07.627269 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9a11f5-9170-4524-bd05-33dd8f0911c5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e158e5c87f4954aeb297f42dabdc16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:07.629653 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a11f5-9170-4524-bd05-33dd8f0911c5 service placement] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:07.630144 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a11f5-9170-4524-bd05-33dd8f0911c5 service placement] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:07.630890 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a11f5-9170-4524-bd05-33dd8f0911c5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:07.636513 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-ed9a11f5-9170-4524-bd05-33dd8f0911c5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:07.683673 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9a11f5-9170-4524-bd05-33dd8f0911c5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:07.690010 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9a11f5-9170-4524-bd05-33dd8f0911c5 service placement] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:07.714358 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 834/1654] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 16:43:09.735042 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20b451-0543-4e89-9dfa-a79849afc991 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:09.735567 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20b451-0543-4e89-9dfa-a79849afc991 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:09.736816 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20b451-0543-4e89-9dfa-a79849afc991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:09.798507 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-4b20b451-0543-4e89-9dfa-a79849afc991 None None] MFA Rules not processed for user `6e88db1bf9294decaf171756772a5066`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:09.800272 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-4b20b451-0543-4e89-9dfa-a79849afc991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:09.947925 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:10.038628 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 821/1655] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:10.045871 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c81570-5a45-49f5-8b33-0eed2377020e None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:10.046793 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c81570-5a45-49f5-8b33-0eed2377020e None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:10.047218 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c81570-5a45-49f5-8b33-0eed2377020e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:10.079305 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-b8c81570-5a45-49f5-8b33-0eed2377020e None None] MFA Rules not processed for user `6e88db1bf9294decaf171756772a5066`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:10.081617 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b8c81570-5a45-49f5-8b33-0eed2377020e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:10.094811 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 835/1656] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:10.106070 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b20b451-0543-4e89-9dfa-a79849afc991 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:10.136177 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b20b451-0543-4e89-9dfa-a79849afc991 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:10.153509 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28972d8-3c18-480b-ab57-04d3f1a56165 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:10.154607 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28972d8-3c18-480b-ab57-04d3f1a56165 service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:10.154607 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28972d8-3c18-480b-ab57-04d3f1a56165 service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:10.154769 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28972d8-3c18-480b-ab57-04d3f1a56165 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:10.179836 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28972d8-3c18-480b-ab57-04d3f1a56165 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:10.196910 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28972d8-3c18-480b-ab57-04d3f1a56165 service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:10.224815 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 822/1657] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:43:11.074718 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f65b110-0e4c-413c-8652-11bd48945f95 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:11.075535 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f65b110-0e4c-413c-8652-11bd48945f95 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:11.076001 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f65b110-0e4c-413c-8652-11bd48945f95 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:11.078966 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 836/1658] 10.222.0.36 () {54 vars in 912 bytes} [Thu May 19 16:43:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:43:11.095655 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5f464-d4f4-4913-9ef7-107944d7cfb2 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:11.096761 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5f464-d4f4-4913-9ef7-107944d7cfb2 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:11.097297 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5f464-d4f4-4913-9ef7-107944d7cfb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:11.158500 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-93a5f464-d4f4-4913-9ef7-107944d7cfb2 None None] MFA Rules not processed for user `33472643b65e4fe08fd5b3aa2c618133`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:11.162788 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-93a5f464-d4f4-4913-9ef7-107944d7cfb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:11.194349 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 823/1659] 10.222.0.36 () {58 vars in 1009 bytes} [Thu May 19 16:43:11 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:11.248290 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f65b110-0e4c-413c-8652-11bd48945f95 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:11.300681 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f65b110-0e4c-413c-8652-11bd48945f95 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:11.340963 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6660aff-7924-4f71-b491-a8862284b718 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:11.344518 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6660aff-7924-4f71-b491-a8862284b718 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:11.345831 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6660aff-7924-4f71-b491-a8862284b718 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:11.347746 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6660aff-7924-4f71-b491-a8862284b718 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:11.413016 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6660aff-7924-4f71-b491-a8862284b718 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:11.418150 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6660aff-7924-4f71-b491-a8862284b718 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:11.444939 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 837/1660] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:43:11.966798 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59bbe3e-8299-4621-857e-204ea03e001a None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:11.967664 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59bbe3e-8299-4621-857e-204ea03e001a None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:11.970091 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59bbe3e-8299-4621-857e-204ea03e001a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:12.110437 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-b59bbe3e-8299-4621-857e-204ea03e001a None None] MFA Rules not processed for user `0695c868a2d844089fcda29ea3653d5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:12.114671 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-b59bbe3e-8299-4621-857e-204ea03e001a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:12.442783 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 824/1661] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 480 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:12.451873 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6660aff-7924-4f71-b491-a8862284b718 service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:12.492952 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6660aff-7924-4f71-b491-a8862284b718 service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:12.513746 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b260a27a-f9dd-485a-9ce6-5937d9022205 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:12.514441 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b260a27a-f9dd-485a-9ce6-5937d9022205 service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:12.514735 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b260a27a-f9dd-485a-9ce6-5937d9022205 service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:12.515021 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b260a27a-f9dd-485a-9ce6-5937d9022205 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:12.533931 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b260a27a-f9dd-485a-9ce6-5937d9022205 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:12.535801 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b260a27a-f9dd-485a-9ce6-5937d9022205 service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:12.567087 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 838/1662] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:43:14.063228 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc13b4-e420-4b27-889c-309e75edbc2f None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:14.064364 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc13b4-e420-4b27-889c-309e75edbc2f None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:14.064364 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc13b4-e420-4b27-889c-309e75edbc2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:14.125480 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-d6cc13b4-e420-4b27-889c-309e75edbc2f None None] MFA Rules not processed for user `6e88db1bf9294decaf171756772a5066`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:14.134969 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-d6cc13b4-e420-4b27-889c-309e75edbc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:14.173448 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 825/1663] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:15.607773 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb56e6e-53a5-48a3-a378-4e8d24d7d8e1 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:15.616427 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb56e6e-53a5-48a3-a378-4e8d24d7d8e1 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:15.617290 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb56e6e-53a5-48a3-a378-4e8d24d7d8e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:15.703058 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-ceb56e6e-53a5-48a3-a378-4e8d24d7d8e1 None None] MFA Rules not processed for user `86b78ffb46894509bf8e81a154073885`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:15.704453 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-ceb56e6e-53a5-48a3-a378-4e8d24d7d8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:15.872305 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 839/1664] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:43:16.121510 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cc13b4-e420-4b27-889c-309e75edbc2f None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:16.159771 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cc13b4-e420-4b27-889c-309e75edbc2f None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:16.202228 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99228d4e-c575-4fb1-b4bf-47038c16f37f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:16.205607 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99228d4e-c575-4fb1-b4bf-47038c16f37f service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:16.205905 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99228d4e-c575-4fb1-b4bf-47038c16f37f service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:16.207501 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99228d4e-c575-4fb1-b4bf-47038c16f37f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:16.254620 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99228d4e-c575-4fb1-b4bf-47038c16f37f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:16.273642 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99228d4e-c575-4fb1-b4bf-47038c16f37f service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:16.330877 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 826/1665] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:16 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:43:16.782877 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bff9a-9081-4a89-8dde-dca81a6f4657 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:16.783782 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bff9a-9081-4a89-8dde-dca81a6f4657 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:16.784218 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bff9a-9081-4a89-8dde-dca81a6f4657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:16.810491 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-856bff9a-9081-4a89-8dde-dca81a6f4657 None None] MFA Rules not processed for user `a30dc1dd2b00463bb3c7957864c56043`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:16.816137 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-856bff9a-9081-4a89-8dde-dca81a6f4657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:17.001636 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 840/1666] 10.222.0.36 () {58 vars in 1009 bytes} [Thu May 19 16:43:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:17.137507 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99228d4e-c575-4fb1-b4bf-47038c16f37f service nova] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:17.253763 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856bff9a-9081-4a89-8dde-dca81a6f4657 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:17.256025 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-856bff9a-9081-4a89-8dde-dca81a6f4657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:17.344514 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99228d4e-c575-4fb1-b4bf-47038c16f37f service nova] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:17.363833 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45190540-971f-426b-8f90-4f950a54e7b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:17.366559 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45190540-971f-426b-8f90-4f950a54e7b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:17.366844 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45190540-971f-426b-8f90-4f950a54e7b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:17.367612 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45190540-971f-426b-8f90-4f950a54e7b0 admin admin] PATH_INFO: `/v3/users/cd00835e169a4c42bea59e8204e7c09e` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:17.371584 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45190540-971f-426b-8f90-4f950a54e7b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd00835e169a4c42bea59e8204e7c09e)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:17.373947 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45190540-971f-426b-8f90-4f950a54e7b0 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:17.419122 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:17.464328 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:17.487167 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856bff9a-9081-4a89-8dde-dca81a6f4657 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:17.506119 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a94906-64c5-41d1-b4d3-3c1f6b7a3ffd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffe7cebc6f3b40d8a8e1866ed2da9822', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:17.509377 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a94906-64c5-41d1-b4d3-3c1f6b7a3ffd service swift] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:17.510255 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a94906-64c5-41d1-b4d3-3c1f6b7a3ffd service swift] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:17.511983 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a94906-64c5-41d1-b4d3-3c1f6b7a3ffd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:17.535107 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-45190540-971f-426b-8f90-4f950a54e7b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd00835e169a4c42bea59e8204e7c09e'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:17.540542 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-45190540-971f-426b-8f90-4f950a54e7b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd00835e169a4c42bea59e8204e7c09e'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:17.568489 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a94906-64c5-41d1-b4d3-3c1f6b7a3ffd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:17.570034 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-45190540-971f-426b-8f90-4f950a54e7b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd00835e169a4c42bea59e8204e7c09e'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:17.583039 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a94906-64c5-41d1-b4d3-3c1f6b7a3ffd service swift] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:17.645477 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 841/1667] 10.222.0.36 () {60 vars in 1384 bytes} [Thu May 19 16:43:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 16:43:17.653964 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-45190540-971f-426b-8f90-4f950a54e7b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd00835e169a4c42bea59e8204e7c09e'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:17.655734 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-45190540-971f-426b-8f90-4f950a54e7b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd00835e169a4c42bea59e8204e7c09e'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:17.747871 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 827/1668] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:43:17 2022] DELETE /identity/v3/users/cd00835e169a4c42bea59e8204e7c09e => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:17.754409 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583a132-4c78-4511-92bd-d244be343f55 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:17.754732 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583a132-4c78-4511-92bd-d244be343f55 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:17.755444 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583a132-4c78-4511-92bd-d244be343f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:17.853014 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-d583a132-4c78-4511-92bd-d244be343f55 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:17.863430 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d583a132-4c78-4511-92bd-d244be343f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:17.944941 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed8c74-3761-4a5b-b255-91ca8dfcfea4 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:17.945817 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed8c74-3761-4a5b-b255-91ca8dfcfea4 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:17.946330 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed8c74-3761-4a5b-b255-91ca8dfcfea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:18.103880 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-c4ed8c74-3761-4a5b-b255-91ca8dfcfea4 None None] MFA Rules not processed for user `0695c868a2d844089fcda29ea3653d5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:18.111441 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-c4ed8c74-3761-4a5b-b255-91ca8dfcfea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:18.386736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 842/1669] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 635 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:18.482041 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 828/1670] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 542 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:19.082109 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d583a132-4c78-4511-92bd-d244be343f55 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:19.085649 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d583a132-4c78-4511-92bd-d244be343f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:19.304898 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d583a132-4c78-4511-92bd-d244be343f55 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:19.309812 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c64c7f-e242-4d05-ab89-4ef8c95b18b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:19.311220 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c64c7f-e242-4d05-ab89-4ef8c95b18b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:19.312073 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c64c7f-e242-4d05-ab89-4ef8c95b18b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:19.312517 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c64c7f-e242-4d05-ab89-4ef8c95b18b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:19.350977 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c64c7f-e242-4d05-ab89-4ef8c95b18b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:19.358697 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c64c7f-e242-4d05-ab89-4ef8c95b18b1 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:19.442954 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 843/1671] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:43:19.869439 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ee70a-d17c-42fa-b98b-a27de14c4a5b None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:19.870160 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ee70a-d17c-42fa-b98b-a27de14c4a5b None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:19.870470 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ee70a-d17c-42fa-b98b-a27de14c4a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:19.941863 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-7c9ee70a-d17c-42fa-b98b-a27de14c4a5b None None] MFA Rules not processed for user `33472643b65e4fe08fd5b3aa2c618133`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:19.943049 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-7c9ee70a-d17c-42fa-b98b-a27de14c4a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:19.961100 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 829/1672] 10.222.0.48 () {58 vars in 1015 bytes} [Thu May 19 16:43:19 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:20.179291 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c64c7f-e242-4d05-ab89-4ef8c95b18b1 service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:20.183311 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-66c64c7f-e242-4d05-ab89-4ef8c95b18b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:20.217579 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c64c7f-e242-4d05-ab89-4ef8c95b18b1 service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:20.228480 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b1931d-8049-4eb4-bd67-944006833c7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:20.231872 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1931d-8049-4eb4-bd67-944006833c7f service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:20.244543 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1931d-8049-4eb4-bd67-944006833c7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:20.245037 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1931d-8049-4eb4-bd67-944006833c7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:20.276033 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b1931d-8049-4eb4-bd67-944006833c7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:20.283058 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b1931d-8049-4eb4-bd67-944006833c7f service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:20.322387 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 844/1673] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:43:22.611518 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ee70a-d17c-42fa-b98b-a27de14c4a5b None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:22.613128 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-7c9ee70a-d17c-42fa-b98b-a27de14c4a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:22.643293 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ee70a-d17c-42fa-b98b-a27de14c4a5b None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:22.656105 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a9f8e0-7089-4995-9247-000c4844b15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:22.662028 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9f8e0-7089-4995-9247-000c4844b15b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:22.663030 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9f8e0-7089-4995-9247-000c4844b15b admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:22.664766 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9f8e0-7089-4995-9247-000c4844b15b admin admin] PATH_INFO: `/v3/users/a7a919d815074ed1aba725a56e198b47` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:22.669502 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a9f8e0-7089-4995-9247-000c4844b15b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7a919d815074ed1aba725a56e198b47)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:22.675987 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a9f8e0-7089-4995-9247-000c4844b15b admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:22.824283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-a0a9f8e0-7089-4995-9247-000c4844b15b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7a919d815074ed1aba725a56e198b47'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:22.831046 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-a0a9f8e0-7089-4995-9247-000c4844b15b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7a919d815074ed1aba725a56e198b47'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:22.889296 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-a0a9f8e0-7089-4995-9247-000c4844b15b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7a919d815074ed1aba725a56e198b47'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:22.944821 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-a0a9f8e0-7089-4995-9247-000c4844b15b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7a919d815074ed1aba725a56e198b47'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:22.946984 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-a0a9f8e0-7089-4995-9247-000c4844b15b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7a919d815074ed1aba725a56e198b47'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:22.954614 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b1931d-8049-4eb4-bd67-944006833c7f service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:22.956717 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-26b1931d-8049-4eb4-bd67-944006833c7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:23.038447 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b1931d-8049-4eb4-bd67-944006833c7f service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:23.061612 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 830/1674] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:43:22 2022] DELETE /identity/v3/users/a7a919d815074ed1aba725a56e198b47 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:23.068280 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d402e39-ceeb-44bc-b298-808365452bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:23.069109 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7de0c-ea0e-4276-bd08-b75b7b5f6079 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:23.069466 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7de0c-ea0e-4276-bd08-b75b7b5f6079 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:23.069748 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7de0c-ea0e-4276-bd08-b75b7b5f6079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:23.071664 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d402e39-ceeb-44bc-b298-808365452bd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:23.071966 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d402e39-ceeb-44bc-b298-808365452bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:23.073474 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d402e39-ceeb-44bc-b298-808365452bd3 admin admin] PATH_INFO: `/v3/projects/e0ed7ea223d744b283eacf61514bc9aa` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:23.077065 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d402e39-ceeb-44bc-b298-808365452bd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0ed7ea223d744b283eacf61514bc9aa)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:23.084663 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d402e39-ceeb-44bc-b298-808365452bd3 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:23.130424 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-5ca7de0c-ea0e-4276-bd08-b75b7b5f6079 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:23.136578 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-5ca7de0c-ea0e-4276-bd08-b75b7b5f6079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:23.191398 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:23.278899 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-7d402e39-ceeb-44bc-b298-808365452bd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0ed7ea223d744b283eacf61514bc9aa', 'request_id': 'req-7d402e39-ceeb-44bc-b298-808365452bd3'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:23.306775 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-7d402e39-ceeb-44bc-b298-808365452bd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0ed7ea223d744b283eacf61514bc9aa', 'request_id': 'req-7d402e39-ceeb-44bc-b298-808365452bd3'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:23.318735 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-7d402e39-ceeb-44bc-b298-808365452bd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0ed7ea223d744b283eacf61514bc9aa', 'request_id': 'req-7d402e39-ceeb-44bc-b298-808365452bd3'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:23.329179 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:23.379939 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-7d402e39-ceeb-44bc-b298-808365452bd3 admin admin] The token cache is being invalidate because project e0ed7ea223d744b283eacf61514bc9aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:43:23.399515 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-7d402e39-ceeb-44bc-b298-808365452bd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:23.415859 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-7d402e39-ceeb-44bc-b298-808365452bd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:23.430024 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 845/1675] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:43:22 2022] DELETE /identity/v3/projects/e0ed7ea223d744b283eacf61514bc9aa => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:23.441440 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98afde-f9fd-47fb-b31a-e80c1274b1ad None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:23.441805 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98afde-f9fd-47fb-b31a-e80c1274b1ad None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:23.442102 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98afde-f9fd-47fb-b31a-e80c1274b1ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:23.487485 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 831/1676] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 422 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:23.538072 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-aa98afde-f9fd-47fb-b31a-e80c1274b1ad None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:23.539324 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-aa98afde-f9fd-47fb-b31a-e80c1274b1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:23.557469 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca7de0c-ea0e-4276-bd08-b75b7b5f6079 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:23.561995 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-5ca7de0c-ea0e-4276-bd08-b75b7b5f6079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:23.772398 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 846/1677] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:23.775009 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa98afde-f9fd-47fb-b31a-e80c1274b1ad None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:23.781351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-aa98afde-f9fd-47fb-b31a-e80c1274b1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:23.811210 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa98afde-f9fd-47fb-b31a-e80c1274b1ad None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:23.825058 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc53663-8467-4def-9ce0-30095c1269d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:23.826148 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc53663-8467-4def-9ce0-30095c1269d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:23.826444 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc53663-8467-4def-9ce0-30095c1269d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:23.826736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc53663-8467-4def-9ce0-30095c1269d9 admin admin] PATH_INFO: `/v3/users/df3b4f232bd9411999e26d6adc77837f` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:23.830510 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc53663-8467-4def-9ce0-30095c1269d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df3b4f232bd9411999e26d6adc77837f)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:23.833169 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: WARNING py.warnings [None req-abc53663-8467-4def-9ce0-30095c1269d9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 16:43:23.833169 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: warnings.warn(msg) May 19 16:43:23.833169 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: May 19 16:43:23.837901 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc53663-8467-4def-9ce0-30095c1269d9 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:24.038524 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca7de0c-ea0e-4276-bd08-b75b7b5f6079 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:24.074702 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-abc53663-8467-4def-9ce0-30095c1269d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df3b4f232bd9411999e26d6adc77837f'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:24.077748 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-abc53663-8467-4def-9ce0-30095c1269d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df3b4f232bd9411999e26d6adc77837f'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:24.080955 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-abc53663-8467-4def-9ce0-30095c1269d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df3b4f232bd9411999e26d6adc77837f'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:24.108356 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-abc53663-8467-4def-9ce0-30095c1269d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df3b4f232bd9411999e26d6adc77837f'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:24.311801 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa275c93-11be-4477-9291-c75dabd630c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:24.325743 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa275c93-11be-4477-9291-c75dabd630c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:24.326239 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa275c93-11be-4477-9291-c75dabd630c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:24.326531 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa275c93-11be-4477-9291-c75dabd630c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:24.342904 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-aa275c93-11be-4477-9291-c75dabd630c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:24.381253 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa275c93-11be-4477-9291-c75dabd630c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:24.384212 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa275c93-11be-4477-9291-c75dabd630c2 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:24.450416 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 832/1678] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 892 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:43:24.454798 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17980f3-6cff-4038-b972-a8d50dbd08a3 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:24.455617 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17980f3-6cff-4038-b972-a8d50dbd08a3 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:24.458197 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17980f3-6cff-4038-b972-a8d50dbd08a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:24.535413 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-b17980f3-6cff-4038-b972-a8d50dbd08a3 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:24.539367 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-b17980f3-6cff-4038-b972-a8d50dbd08a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:24.566583 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 833/1679] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:24.572262 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17980f3-6cff-4038-b972-a8d50dbd08a3 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:24.601799 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17980f3-6cff-4038-b972-a8d50dbd08a3 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:24.609430 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae92ec75-3933-4771-a45b-d5c23737be8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:24.610573 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92ec75-3933-4771-a45b-d5c23737be8c admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:24.611053 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92ec75-3933-4771-a45b-d5c23737be8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:24.611481 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92ec75-3933-4771-a45b-d5c23737be8c admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:24.612626 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae92ec75-3933-4771-a45b-d5c23737be8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:43:24.613133 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae92ec75-3933-4771-a45b-d5c23737be8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:24.615381 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae92ec75-3933-4771-a45b-d5c23737be8c admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:24.649403 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 834/1680] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:43:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:43:24.654075 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae92ec75-3933-4771-a45b-d5c23737be8c admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:24.680731 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae92ec75-3933-4771-a45b-d5c23737be8c admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:24.686002 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-abc53663-8467-4def-9ce0-30095c1269d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df3b4f232bd9411999e26d6adc77837f'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:24.695087 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f93737-5f94-4181-9d93-e074e20f6bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:24.697382 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f93737-5f94-4181-9d93-e074e20f6bad admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:24.698559 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f93737-5f94-4181-9d93-e074e20f6bad admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:24.698912 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f93737-5f94-4181-9d93-e074e20f6bad admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:24.701057 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f93737-5f94-4181-9d93-e074e20f6bad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:24.708823 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f93737-5f94-4181-9d93-e074e20f6bad admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:24.728081 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 847/1681] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:43:23 2022] DELETE /identity/v3/users/df3b4f232bd9411999e26d6adc77837f => generated 0 bytes in 955 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:24.735962 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f9bb9-728e-4828-bc3b-69f3036ea32a None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:24.736264 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f9bb9-728e-4828-bc3b-69f3036ea32a None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:24.736531 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f9bb9-728e-4828-bc3b-69f3036ea32a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:24.786718 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-6a3f9bb9-728e-4828-bc3b-69f3036ea32a None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:24.795497 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-6a3f9bb9-728e-4828-bc3b-69f3036ea32a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:24.846423 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 835/1682] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:43:24 2022] POST /identity/v3/projects => generated 365 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:43:24.849768 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f93737-5f94-4181-9d93-e074e20f6bad admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:25.059211 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:25.128147 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 848/1683] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:43:25.157052 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f93737-5f94-4181-9d93-e074e20f6bad admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:25.169590 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb59dc6-c690-45ef-b80a-d8217196eb59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:25.170535 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb59dc6-c690-45ef-b80a-d8217196eb59 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:25.171533 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb59dc6-c690-45ef-b80a-d8217196eb59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:25.171967 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb59dc6-c690-45ef-b80a-d8217196eb59 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:25.174617 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb59dc6-c690-45ef-b80a-d8217196eb59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:25.176771 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb59dc6-c690-45ef-b80a-d8217196eb59 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:25.234292 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 836/1684] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:43:24 2022] POST /identity/v3/users => generated 283 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:43:25.237688 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a435b-e62c-4f9a-a1e3-87f86bcd21bb None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:25.238374 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a435b-e62c-4f9a-a1e3-87f86bcd21bb None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:25.239059 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb59dc6-c690-45ef-b80a-d8217196eb59 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:25.239726 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a435b-e62c-4f9a-a1e3-87f86bcd21bb None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:25.243475 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 849/1685] 10.222.0.36 () {54 vars in 921 bytes} [Thu May 19 16:43:25 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:43:25.279642 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb59dc6-c690-45ef-b80a-d8217196eb59 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:25.296852 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a435b-e62c-4f9a-a1e3-87f86bcd21bb None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:25.299909 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-818a435b-e62c-4f9a-a1e3-87f86bcd21bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:25.302957 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320878d8-a683-434d-abf6-6bcba455f211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:25.309518 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320878d8-a683-434d-abf6-6bcba455f211 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:25.311524 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320878d8-a683-434d-abf6-6bcba455f211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:25.312064 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320878d8-a683-434d-abf6-6bcba455f211 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:25.317578 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320878d8-a683-434d-abf6-6bcba455f211 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:25.322132 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320878d8-a683-434d-abf6-6bcba455f211 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:25.348448 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:25.383990 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 837/1686] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:25 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:43:25.388659 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320878d8-a683-434d-abf6-6bcba455f211 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:25.441063 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320878d8-a683-434d-abf6-6bcba455f211 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:25.473840 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ef28e-7f23-4fef-bcb3-cb077fc21851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:25.474706 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ef28e-7f23-4fef-bcb3-cb077fc21851 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:25.475170 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ef28e-7f23-4fef-bcb3-cb077fc21851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:25.480148 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ef28e-7f23-4fef-bcb3-cb077fc21851 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:25.483782 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1ef28e-7f23-4fef-bcb3-cb077fc21851 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:25.486760 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1ef28e-7f23-4fef-bcb3-cb077fc21851 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:25.521598 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 838/1687] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:25 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:43:25.525791 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d306d-934b-46b0-9f21-46d576c3e972 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:25.526209 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d306d-934b-46b0-9f21-46d576c3e972 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:25.526417 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d306d-934b-46b0-9f21-46d576c3e972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:25.528257 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a435b-e62c-4f9a-a1e3-87f86bcd21bb None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:25.533864 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f46357-6c27-4394-8351-1e018d6ddb9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:25.534601 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f46357-6c27-4394-8351-1e018d6ddb9c service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:25.534891 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f46357-6c27-4394-8351-1e018d6ddb9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:25.535179 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f46357-6c27-4394-8351-1e018d6ddb9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:25.551784 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f46357-6c27-4394-8351-1e018d6ddb9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:25.557660 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f46357-6c27-4394-8351-1e018d6ddb9c service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:25.571036 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-e74d306d-934b-46b0-9f21-46d576c3e972 None None] MFA Rules not processed for user `58bdeada00b74b7b8e186e6dc51e6dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:25.574091 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-e74d306d-934b-46b0-9f21-46d576c3e972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:25.581489 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 850/1688] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:43:25.582875 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f46357-6c27-4394-8351-1e018d6ddb9c service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:25.599328 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f46357-6c27-4394-8351-1e018d6ddb9c service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:25.612915 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017c5fe5-385b-49ef-9927-a602d6cb16ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:25.613986 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c5fe5-385b-49ef-9927-a602d6cb16ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:25.614370 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c5fe5-385b-49ef-9927-a602d6cb16ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:25.614673 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c5fe5-385b-49ef-9927-a602d6cb16ba admin admin] PATH_INFO: `/v3/projects/3197c37a5080426bae587174668b496d/users/4a8961c2ccef4a5c9d73560037a23a60/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:25.647480 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017c5fe5-385b-49ef-9927-a602d6cb16ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=3197c37a5080426bae587174668b496d, user_id=4a8961c2ccef4a5c9d73560037a23a60, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:25.650579 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017c5fe5-385b-49ef-9927-a602d6cb16ba admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:25.687192 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 851/1689] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:43:25 2022] PUT /identity/v3/projects/3197c37a5080426bae587174668b496d/users/4a8961c2ccef4a5c9d73560037a23a60/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:25.732007 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac5b6b-b0e7-4a0e-94b4-4eb0db983cf8 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:25.732308 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac5b6b-b0e7-4a0e-94b4-4eb0db983cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:25.732580 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac5b6b-b0e7-4a0e-94b4-4eb0db983cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:25.769661 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 839/1690] 10.222.0.36 () {58 vars in 1018 bytes} [Thu May 19 16:43:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:25.810036 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-55ac5b6b-b0e7-4a0e-94b4-4eb0db983cf8 None None] MFA Rules not processed for user `4a8961c2ccef4a5c9d73560037a23a60`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:25.811382 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-55ac5b6b-b0e7-4a0e-94b4-4eb0db983cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:25.818488 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d306d-934b-46b0-9f21-46d576c3e972 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:25.819949 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-e74d306d-934b-46b0-9f21-46d576c3e972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:25.838061 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d306d-934b-46b0-9f21-46d576c3e972 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:26.006954 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb721b0-4a90-437f-971d-e200a2ed15c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:26.009987 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb721b0-4a90-437f-971d-e200a2ed15c1 service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:26.010726 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb721b0-4a90-437f-971d-e200a2ed15c1 service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:26.011504 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb721b0-4a90-437f-971d-e200a2ed15c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:26.044476 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb721b0-4a90-437f-971d-e200a2ed15c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:26.046156 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb721b0-4a90-437f-971d-e200a2ed15c1 service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:26.060627 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 852/1691] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:25 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:26.082898 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac5b6b-b0e7-4a0e-94b4-4eb0db983cf8 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:26.084515 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-55ac5b6b-b0e7-4a0e-94b4-4eb0db983cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:26.099207 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 840/1692] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 16:43:26.408250 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac5b6b-b0e7-4a0e-94b4-4eb0db983cf8 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:26.419552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0665a1-bb33-46b0-b0cd-1ff9bd93aa15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:26.423839 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0665a1-bb33-46b0-b0cd-1ff9bd93aa15 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:26.426674 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0665a1-bb33-46b0-b0cd-1ff9bd93aa15 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:26.427379 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0665a1-bb33-46b0-b0cd-1ff9bd93aa15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:26.464851 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0665a1-bb33-46b0-b0cd-1ff9bd93aa15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:26.467029 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0665a1-bb33-46b0-b0cd-1ff9bd93aa15 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:26.513312 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 853/1693] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:26 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:43:26.670182 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb721b0-4a90-437f-971d-e200a2ed15c1 service nova] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:26.676104 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-8bb721b0-4a90-437f-971d-e200a2ed15c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:26.752600 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:26.979450 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb721b0-4a90-437f-971d-e200a2ed15c1 service nova] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:26.997424 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b275ec53-c6ff-4b25-a7dc-18ffd5b1e3fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:27.003522 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b275ec53-c6ff-4b25-a7dc-18ffd5b1e3fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:27.004004 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b275ec53-c6ff-4b25-a7dc-18ffd5b1e3fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:27.004605 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b275ec53-c6ff-4b25-a7dc-18ffd5b1e3fa admin admin] PATH_INFO: `/v3/projects/5a6eafe486fe4b5cadac660a921721c6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:27.008914 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b275ec53-c6ff-4b25-a7dc-18ffd5b1e3fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a6eafe486fe4b5cadac660a921721c6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:27.019055 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: WARNING py.warnings [None req-b275ec53-c6ff-4b25-a7dc-18ffd5b1e3fa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 16:43:27.019055 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: warnings.warn(msg) May 19 16:43:27.019055 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: May 19 16:43:27.024199 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b275ec53-c6ff-4b25-a7dc-18ffd5b1e3fa admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:27.229807 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b275ec53-c6ff-4b25-a7dc-18ffd5b1e3fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a6eafe486fe4b5cadac660a921721c6', 'request_id': 'req-b275ec53-c6ff-4b25-a7dc-18ffd5b1e3fa'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:27.249704 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b275ec53-c6ff-4b25-a7dc-18ffd5b1e3fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a6eafe486fe4b5cadac660a921721c6', 'request_id': 'req-b275ec53-c6ff-4b25-a7dc-18ffd5b1e3fa'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:27.280160 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b275ec53-c6ff-4b25-a7dc-18ffd5b1e3fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a6eafe486fe4b5cadac660a921721c6', 'request_id': 'req-b275ec53-c6ff-4b25-a7dc-18ffd5b1e3fa'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:27.312574 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b275ec53-c6ff-4b25-a7dc-18ffd5b1e3fa admin admin] The token cache is being invalidate because project 5a6eafe486fe4b5cadac660a921721c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:43:27.313004 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b275ec53-c6ff-4b25-a7dc-18ffd5b1e3fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:27.313867 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b275ec53-c6ff-4b25-a7dc-18ffd5b1e3fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:27.320218 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 841/1694] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:43:26 2022] DELETE /identity/v3/projects/5a6eafe486fe4b5cadac660a921721c6 => generated 0 bytes in 648 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:27.322381 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577ed8e-a761-4c8c-95c8-6764ae12354b None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:27.322794 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577ed8e-a761-4c8c-95c8-6764ae12354b None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:27.323188 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577ed8e-a761-4c8c-95c8-6764ae12354b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:27.383038 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-9577ed8e-a761-4c8c-95c8-6764ae12354b None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:27.385996 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-9577ed8e-a761-4c8c-95c8-6764ae12354b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:27.439367 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:27.600799 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 854/1695] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:27.610152 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd4bfcf-9b14-4cd1-b1c6-3c890b005591 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:27.610627 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd4bfcf-9b14-4cd1-b1c6-3c890b005591 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:27.611030 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd4bfcf-9b14-4cd1-b1c6-3c890b005591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:27.653417 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-cbd4bfcf-9b14-4cd1-b1c6-3c890b005591 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:27.660796 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-cbd4bfcf-9b14-4cd1-b1c6-3c890b005591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:27.679590 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 842/1696] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:27.684709 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9577ed8e-a761-4c8c-95c8-6764ae12354b None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:27.701626 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9577ed8e-a761-4c8c-95c8-6764ae12354b None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:27.717948 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8aafb0-3a12-461d-839c-6d83b4cd65ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:27.720424 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8aafb0-3a12-461d-839c-6d83b4cd65ca admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:27.721089 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8aafb0-3a12-461d-839c-6d83b4cd65ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:27.721969 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8aafb0-3a12-461d-839c-6d83b4cd65ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:27.725386 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8aafb0-3a12-461d-839c-6d83b4cd65ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:43:27.725797 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8aafb0-3a12-461d-839c-6d83b4cd65ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:27.732129 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8aafb0-3a12-461d-839c-6d83b4cd65ca admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:27.771495 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302bc389-c853-4f72-953a-1ab96acd395e None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:27.772291 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302bc389-c853-4f72-953a-1ab96acd395e None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:27.772858 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302bc389-c853-4f72-953a-1ab96acd395e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:27.783489 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 855/1697] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:43:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:43:27.788147 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8aafb0-3a12-461d-839c-6d83b4cd65ca admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:27.813901 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8aafb0-3a12-461d-839c-6d83b4cd65ca admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:27.825993 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b17ee0e-da5c-4847-afb6-fff1d883a1c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:27.826692 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17ee0e-da5c-4847-afb6-fff1d883a1c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:27.827014 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17ee0e-da5c-4847-afb6-fff1d883a1c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:27.827303 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17ee0e-da5c-4847-afb6-fff1d883a1c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:27.828128 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b17ee0e-da5c-4847-afb6-fff1d883a1c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:27.830397 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b17ee0e-da5c-4847-afb6-fff1d883a1c4 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:27.834999 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-302bc389-c853-4f72-953a-1ab96acd395e None None] MFA Rules not processed for user `86b78ffb46894509bf8e81a154073885`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:27.838524 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-302bc389-c853-4f72-953a-1ab96acd395e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:27.914291 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 856/1698] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:43:27 2022] POST /identity/v3/projects => generated 403 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:43:27.921694 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b17ee0e-da5c-4847-afb6-fff1d883a1c4 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:28.016125 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 843/1699] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:28.167345 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b17ee0e-da5c-4847-afb6-fff1d883a1c4 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:28.189928 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a80df6-93c6-43dd-a4a7-51bf7ade2c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:28.192450 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a80df6-93c6-43dd-a4a7-51bf7ade2c7b admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:28.193991 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a80df6-93c6-43dd-a4a7-51bf7ade2c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:28.195678 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a80df6-93c6-43dd-a4a7-51bf7ade2c7b admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:28.197999 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a80df6-93c6-43dd-a4a7-51bf7ade2c7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:28.202202 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a80df6-93c6-43dd-a4a7-51bf7ade2c7b admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:28.261191 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 857/1700] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:43:27 2022] POST /identity/v3/users => generated 302 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:43:28.265296 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302bc389-c853-4f72-953a-1ab96acd395e None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:28.298677 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302bc389-c853-4f72-953a-1ab96acd395e None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:28.309289 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09764055-3595-432a-b973-840d6a38601c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:28.309991 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09764055-3595-432a-b973-840d6a38601c admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:28.310286 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09764055-3595-432a-b973-840d6a38601c admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:28.310567 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09764055-3595-432a-b973-840d6a38601c admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:28.311329 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09764055-3595-432a-b973-840d6a38601c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:28.314092 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09764055-3595-432a-b973-840d6a38601c admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:28.351565 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 844/1701] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:28 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:43:28.355782 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a80df6-93c6-43dd-a4a7-51bf7ade2c7b admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:28.372358 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a80df6-93c6-43dd-a4a7-51bf7ade2c7b admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:28.399848 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad44668-a6d0-47cf-afed-4a06c9f93cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:28.402688 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad44668-a6d0-47cf-afed-4a06c9f93cf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:28.403738 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad44668-a6d0-47cf-afed-4a06c9f93cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:28.405306 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad44668-a6d0-47cf-afed-4a06c9f93cf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:28.412820 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad44668-a6d0-47cf-afed-4a06c9f93cf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:28.422630 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad44668-a6d0-47cf-afed-4a06c9f93cf1 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:28.467482 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 858/1702] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:28 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:43:28.468701 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09764055-3595-432a-b973-840d6a38601c admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:28.506761 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09764055-3595-432a-b973-840d6a38601c admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:28.541455 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64aa7e6-5e04-4dd1-9863-50d65c0e1bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:28.542471 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64aa7e6-5e04-4dd1-9863-50d65c0e1bff admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:28.542776 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64aa7e6-5e04-4dd1-9863-50d65c0e1bff admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:28.543059 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64aa7e6-5e04-4dd1-9863-50d65c0e1bff admin admin] PATH_INFO: `/v3/projects/30c09720daa44043b23e1c7e69012537/users/0e8043294f8e44ec8d640d64abe44004/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:28.615485 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64aa7e6-5e04-4dd1-9863-50d65c0e1bff admin admin] RBAC: Authorizing `identity:create_grant(project_id=30c09720daa44043b23e1c7e69012537, user_id=0e8043294f8e44ec8d640d64abe44004, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:28.627709 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64aa7e6-5e04-4dd1-9863-50d65c0e1bff admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:28.677537 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 845/1703] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:43:28 2022] PUT /identity/v3/projects/30c09720daa44043b23e1c7e69012537/users/0e8043294f8e44ec8d640d64abe44004/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:43:28.683219 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6943e-947e-4ac0-9348-df9eeadcde2c None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:28.683832 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6943e-947e-4ac0-9348-df9eeadcde2c None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:28.684265 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6943e-947e-4ac0-9348-df9eeadcde2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:28.734621 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-dbd6943e-947e-4ac0-9348-df9eeadcde2c None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:28.740586 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-dbd6943e-947e-4ac0-9348-df9eeadcde2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:28.905323 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 859/1704] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:28.942021 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64aa7e6-5e04-4dd1-9863-50d65c0e1bff admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:28.951420 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-f64aa7e6-5e04-4dd1-9863-50d65c0e1bff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:28.977999 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:29.098685 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64aa7e6-5e04-4dd1-9863-50d65c0e1bff admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:29.103061 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4344c19-b2f1-4401-8652-380cd4b0d628 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:29.104421 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4344c19-b2f1-4401-8652-380cd4b0d628 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:29.104819 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4344c19-b2f1-4401-8652-380cd4b0d628 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:29.105103 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4344c19-b2f1-4401-8652-380cd4b0d628 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:29.118116 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4344c19-b2f1-4401-8652-380cd4b0d628 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:29.119100 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4344c19-b2f1-4401-8652-380cd4b0d628 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:29.144242 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 846/1705] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:43:29.726537 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd6943e-947e-4ac0-9348-df9eeadcde2c None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:29.728016 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-dbd6943e-947e-4ac0-9348-df9eeadcde2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:29.755146 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:29.764165 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd6943e-947e-4ac0-9348-df9eeadcde2c None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:29.795531 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec46bcf6-97ad-46a5-b9d7-689959a507dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:29.800799 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46bcf6-97ad-46a5-b9d7-689959a507dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:29.801954 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46bcf6-97ad-46a5-b9d7-689959a507dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:29.802387 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46bcf6-97ad-46a5-b9d7-689959a507dd admin admin] PATH_INFO: `/v3/projects/87e169cd9e5e4cc3b391cdb01d11520d` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:29.807679 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec46bcf6-97ad-46a5-b9d7-689959a507dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=87e169cd9e5e4cc3b391cdb01d11520d)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:29.819045 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec46bcf6-97ad-46a5-b9d7-689959a507dd admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:30.069687 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ec46bcf6-97ad-46a5-b9d7-689959a507dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87e169cd9e5e4cc3b391cdb01d11520d', 'request_id': 'req-ec46bcf6-97ad-46a5-b9d7-689959a507dd'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:30.093628 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ec46bcf6-97ad-46a5-b9d7-689959a507dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87e169cd9e5e4cc3b391cdb01d11520d', 'request_id': 'req-ec46bcf6-97ad-46a5-b9d7-689959a507dd'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:30.114316 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ec46bcf6-97ad-46a5-b9d7-689959a507dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87e169cd9e5e4cc3b391cdb01d11520d', 'request_id': 'req-ec46bcf6-97ad-46a5-b9d7-689959a507dd'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:30.162362 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ec46bcf6-97ad-46a5-b9d7-689959a507dd admin admin] The token cache is being invalidate because project 87e169cd9e5e4cc3b391cdb01d11520d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:43:30.163197 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ec46bcf6-97ad-46a5-b9d7-689959a507dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:30.172439 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ec46bcf6-97ad-46a5-b9d7-689959a507dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:30.182208 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4344c19-b2f1-4401-8652-380cd4b0d628 service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:30.183451 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 860/1706] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:43:29 2022] DELETE /identity/v3/projects/87e169cd9e5e4cc3b391cdb01d11520d => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:30.195149 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4344c19-b2f1-4401-8652-380cd4b0d628 service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:30.295951 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2763a7be-f3af-48f7-acf5-aea4734c7d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:30.301407 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2763a7be-f3af-48f7-acf5-aea4734c7d87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:30.301407 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2763a7be-f3af-48f7-acf5-aea4734c7d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:30.301407 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2763a7be-f3af-48f7-acf5-aea4734c7d87 admin admin] PATH_INFO: `/v3/users/219fc44a279147068b3aa624dd832b1e` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:30.301407 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2763a7be-f3af-48f7-acf5-aea4734c7d87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=219fc44a279147068b3aa624dd832b1e)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:30.301407 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2763a7be-f3af-48f7-acf5-aea4734c7d87 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:30.433018 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-2763a7be-f3af-48f7-acf5-aea4734c7d87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '219fc44a279147068b3aa624dd832b1e'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:30.437100 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-2763a7be-f3af-48f7-acf5-aea4734c7d87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '219fc44a279147068b3aa624dd832b1e'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:30.465845 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-2763a7be-f3af-48f7-acf5-aea4734c7d87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '219fc44a279147068b3aa624dd832b1e'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:30.502037 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-2763a7be-f3af-48f7-acf5-aea4734c7d87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '219fc44a279147068b3aa624dd832b1e'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:30.506024 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-2763a7be-f3af-48f7-acf5-aea4734c7d87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '219fc44a279147068b3aa624dd832b1e'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:30.541206 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 847/1707] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:43:30 2022] DELETE /identity/v3/users/219fc44a279147068b3aa624dd832b1e => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:31.818598 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec46bcf6-97ad-46a5-b9d7-689959a507dd admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:31.820068 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-ec46bcf6-97ad-46a5-b9d7-689959a507dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:31.842014 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec46bcf6-97ad-46a5-b9d7-689959a507dd admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:32.049083 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67077447-f2dc-4dff-b3fa-e770dbd8634a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:32.050041 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67077447-f2dc-4dff-b3fa-e770dbd8634a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:32.050509 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67077447-f2dc-4dff-b3fa-e770dbd8634a admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:32.051056 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67077447-f2dc-4dff-b3fa-e770dbd8634a admin admin] PATH_INFO: `/v3/users/00680c852f4a4592bc1c7a5e9cbcbdec` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:32.054942 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67077447-f2dc-4dff-b3fa-e770dbd8634a admin admin] RBAC: Authorizing `identity:delete_user(user_id=00680c852f4a4592bc1c7a5e9cbcbdec)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:32.064810 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67077447-f2dc-4dff-b3fa-e770dbd8634a admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:32.220564 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-67077447-f2dc-4dff-b3fa-e770dbd8634a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00680c852f4a4592bc1c7a5e9cbcbdec'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:32.221585 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-67077447-f2dc-4dff-b3fa-e770dbd8634a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00680c852f4a4592bc1c7a5e9cbcbdec'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:32.229442 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-67077447-f2dc-4dff-b3fa-e770dbd8634a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00680c852f4a4592bc1c7a5e9cbcbdec'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:32.259653 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-67077447-f2dc-4dff-b3fa-e770dbd8634a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00680c852f4a4592bc1c7a5e9cbcbdec'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:32.332499 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-67077447-f2dc-4dff-b3fa-e770dbd8634a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00680c852f4a4592bc1c7a5e9cbcbdec'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:32.396254 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 861/1708] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:43:31 2022] DELETE /identity/v3/users/00680c852f4a4592bc1c7a5e9cbcbdec => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:32.398230 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e48c5-d2fc-4fa1-8726-4d4f5cdc5735 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:32.398806 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e48c5-d2fc-4fa1-8726-4d4f5cdc5735 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:32.398919 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e48c5-d2fc-4fa1-8726-4d4f5cdc5735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:32.435071 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-085e48c5-d2fc-4fa1-8726-4d4f5cdc5735 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:32.436535 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-085e48c5-d2fc-4fa1-8726-4d4f5cdc5735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:32.464924 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde8cc3-4d2e-4cab-bd6b-535665589b7f None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:32.465654 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde8cc3-4d2e-4cab-bd6b-535665589b7f None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:32.466094 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde8cc3-4d2e-4cab-bd6b-535665589b7f None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:32.468538 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 862/1709] 10.222.0.36 () {54 vars in 921 bytes} [Thu May 19 16:43:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 16:43:32.600578 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:32.758522 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 848/1710] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:32.764741 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a7c9a-31e5-4367-b3a6-3a57e0f2e855 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:32.767196 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a7c9a-31e5-4367-b3a6-3a57e0f2e855 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:32.769038 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a7c9a-31e5-4367-b3a6-3a57e0f2e855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:32.807756 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:32.825659 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-675a7c9a-31e5-4367-b3a6-3a57e0f2e855 None None] MFA Rules not processed for user `58bdeada00b74b7b8e186e6dc51e6dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:32.827068 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-675a7c9a-31e5-4367-b3a6-3a57e0f2e855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:32.876786 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085e48c5-d2fc-4fa1-8726-4d4f5cdc5735 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:32.876786 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-085e48c5-d2fc-4fa1-8726-4d4f5cdc5735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:33.134824 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 863/1711] 10.222.0.36 () {58 vars in 1018 bytes} [Thu May 19 16:43:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:33.216825 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085e48c5-d2fc-4fa1-8726-4d4f5cdc5735 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:33.240355 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b5d2f-4d1b-4730-a2be-2881d19e9cf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:33.241335 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b5d2f-4d1b-4730-a2be-2881d19e9cf6 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:33.241748 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b5d2f-4d1b-4730-a2be-2881d19e9cf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:33.242158 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b5d2f-4d1b-4730-a2be-2881d19e9cf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:33.287805 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44b5d2f-4d1b-4730-a2be-2881d19e9cf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:33.289411 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44b5d2f-4d1b-4730-a2be-2881d19e9cf6 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:33.305686 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 849/1712] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:43:33.584052 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a7c9a-31e5-4367-b3a6-3a57e0f2e855 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:33.586017 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-675a7c9a-31e5-4367-b3a6-3a57e0f2e855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:33.601599 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a7c9a-31e5-4367-b3a6-3a57e0f2e855 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:33.606292 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8284f154-aae0-4daf-8264-dacfcdc05075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:33.607181 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8284f154-aae0-4daf-8264-dacfcdc05075 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:33.607610 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8284f154-aae0-4daf-8264-dacfcdc05075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:33.607911 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8284f154-aae0-4daf-8264-dacfcdc05075 admin admin] PATH_INFO: `/v3/projects/384a0628899d4716a7a1b0a7a54523be` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:33.609140 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8284f154-aae0-4daf-8264-dacfcdc05075 admin admin] RBAC: Authorizing `identity:delete_project(project_id=384a0628899d4716a7a1b0a7a54523be)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:33.611882 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8284f154-aae0-4daf-8264-dacfcdc05075 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:33.679635 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b5d2f-4d1b-4730-a2be-2881d19e9cf6 service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:33.682854 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-a44b5d2f-4d1b-4730-a2be-2881d19e9cf6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:33.683234 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-8284f154-aae0-4daf-8264-dacfcdc05075 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '384a0628899d4716a7a1b0a7a54523be', 'request_id': 'req-8284f154-aae0-4daf-8264-dacfcdc05075'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:33.690800 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-8284f154-aae0-4daf-8264-dacfcdc05075 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '384a0628899d4716a7a1b0a7a54523be', 'request_id': 'req-8284f154-aae0-4daf-8264-dacfcdc05075'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:33.699345 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-8284f154-aae0-4daf-8264-dacfcdc05075 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '384a0628899d4716a7a1b0a7a54523be', 'request_id': 'req-8284f154-aae0-4daf-8264-dacfcdc05075'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:33.718762 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b5d2f-4d1b-4730-a2be-2881d19e9cf6 service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:33.722219 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-8284f154-aae0-4daf-8264-dacfcdc05075 admin admin] The token cache is being invalidate because project 384a0628899d4716a7a1b0a7a54523be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:43:33.722519 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-8284f154-aae0-4daf-8264-dacfcdc05075 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:33.723372 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-8284f154-aae0-4daf-8264-dacfcdc05075 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:33.725493 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 864/1713] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:43:33 2022] DELETE /identity/v3/projects/384a0628899d4716a7a1b0a7a54523be => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:33.730617 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539cf57-08e4-4ae1-8dd5-eb0f06d62c40 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:33.730914 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539cf57-08e4-4ae1-8dd5-eb0f06d62c40 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:33.731198 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539cf57-08e4-4ae1-8dd5-eb0f06d62c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:33.765349 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-d539cf57-08e4-4ae1-8dd5-eb0f06d62c40 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:33.767507 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d539cf57-08e4-4ae1-8dd5-eb0f06d62c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:33.855063 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cabf64-459c-4beb-8a72-6008f6ea100c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:33.856405 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cabf64-459c-4beb-8a72-6008f6ea100c service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:33.857048 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cabf64-459c-4beb-8a72-6008f6ea100c service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:33.857731 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cabf64-459c-4beb-8a72-6008f6ea100c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:33.862234 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-05cabf64-459c-4beb-8a72-6008f6ea100c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:33.862393 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 865/1714] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:33.873229 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1cb0be-f696-42a1-ac32-5797eef84418 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:33.873953 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1cb0be-f696-42a1-ac32-5797eef84418 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:33.877066 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1cb0be-f696-42a1-ac32-5797eef84418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:33.898517 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cabf64-459c-4beb-8a72-6008f6ea100c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:33.903376 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cabf64-459c-4beb-8a72-6008f6ea100c service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:33.925189 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 850/1715] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 16:43:33.928842 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-ca1cb0be-f696-42a1-ac32-5797eef84418 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:33.930880 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-ca1cb0be-f696-42a1-ac32-5797eef84418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:33.942224 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 866/1716] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:33.948929 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cabf64-459c-4beb-8a72-6008f6ea100c service nova] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:33.971982 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cabf64-459c-4beb-8a72-6008f6ea100c service nova] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:33.979025 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71d141e-de1d-449e-9690-05cc4f2da738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:33.979937 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d141e-de1d-449e-9690-05cc4f2da738 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:33.980247 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d141e-de1d-449e-9690-05cc4f2da738 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:33.980576 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d141e-de1d-449e-9690-05cc4f2da738 admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:33.981463 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71d141e-de1d-449e-9690-05cc4f2da738 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:43:33.981968 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71d141e-de1d-449e-9690-05cc4f2da738 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:33.983683 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71d141e-de1d-449e-9690-05cc4f2da738 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:34.005571 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 851/1717] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:43:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:43:34.008277 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1cb0be-f696-42a1-ac32-5797eef84418 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:34.024869 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1cb0be-f696-42a1-ac32-5797eef84418 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:34.031523 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef7e9cf-8615-43e8-8a7a-5942ea868b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:34.032584 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7e9cf-8615-43e8-8a7a-5942ea868b24 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:34.033127 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7e9cf-8615-43e8-8a7a-5942ea868b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:34.033527 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7e9cf-8615-43e8-8a7a-5942ea868b24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:34.034627 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef7e9cf-8615-43e8-8a7a-5942ea868b24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:34.037462 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef7e9cf-8615-43e8-8a7a-5942ea868b24 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:34.055443 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:34.082039 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 867/1718] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:43:34 2022] POST /identity/v3/projects => generated 385 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:43:34.086143 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71d141e-de1d-449e-9690-05cc4f2da738 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:34.101634 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71d141e-de1d-449e-9690-05cc4f2da738 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:34.157843 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:34.176744 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99481478-93ce-4230-b81f-c45657b15912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:34.177776 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99481478-93ce-4230-b81f-c45657b15912 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:34.178188 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99481478-93ce-4230-b81f-c45657b15912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:34.178664 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99481478-93ce-4230-b81f-c45657b15912 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:34.184826 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99481478-93ce-4230-b81f-c45657b15912 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:34.189602 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99481478-93ce-4230-b81f-c45657b15912 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:34.225044 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 852/1719] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:43:34 2022] POST /identity/v3/users => generated 293 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:43:34.226198 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef7e9cf-8615-43e8-8a7a-5942ea868b24 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:34.238383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef7e9cf-8615-43e8-8a7a-5942ea868b24 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:34.244226 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f0789c-7919-4f8c-82e4-fc8c033c6418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:34.244999 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f0789c-7919-4f8c-82e4-fc8c033c6418 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:34.245389 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f0789c-7919-4f8c-82e4-fc8c033c6418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:34.245733 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f0789c-7919-4f8c-82e4-fc8c033c6418 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:34.246516 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f0789c-7919-4f8c-82e4-fc8c033c6418 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:34.248207 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f0789c-7919-4f8c-82e4-fc8c033c6418 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:34.267695 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 868/1720] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:34 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:43:34.268992 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99481478-93ce-4230-b81f-c45657b15912 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:34.291198 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99481478-93ce-4230-b81f-c45657b15912 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:34.297027 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61674f5f-e012-4830-a595-6499eb62c5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:34.299071 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61674f5f-e012-4830-a595-6499eb62c5c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:34.299362 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61674f5f-e012-4830-a595-6499eb62c5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:34.299686 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61674f5f-e012-4830-a595-6499eb62c5c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:34.300515 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61674f5f-e012-4830-a595-6499eb62c5c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:34.304393 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61674f5f-e012-4830-a595-6499eb62c5c3 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:34.323880 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 853/1721] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:34 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:43:34.328220 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f0789c-7919-4f8c-82e4-fc8c033c6418 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:34.342519 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f0789c-7919-4f8c-82e4-fc8c033c6418 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:34.356405 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b204d3-030e-4417-832b-933a3d032fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:34.357575 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b204d3-030e-4417-832b-933a3d032fc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:34.357863 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b204d3-030e-4417-832b-933a3d032fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:34.358599 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b204d3-030e-4417-832b-933a3d032fc8 admin admin] PATH_INFO: `/v3/projects/a03897e0f056406dbb5a586e9dc47d2b/users/74629a2eca3b4266a90d72fe4d43ded6/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:34.397800 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc368e3f-d445-4334-b438-04466ab5b354 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:34.398093 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc368e3f-d445-4334-b438-04466ab5b354 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:34.398376 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc368e3f-d445-4334-b438-04466ab5b354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:34.419621 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b204d3-030e-4417-832b-933a3d032fc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a03897e0f056406dbb5a586e9dc47d2b, user_id=74629a2eca3b4266a90d72fe4d43ded6, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:34.423876 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b204d3-030e-4417-832b-933a3d032fc8 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:34.453065 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-bc368e3f-d445-4334-b438-04466ab5b354 None None] MFA Rules not processed for user `0e8043294f8e44ec8d640d64abe44004`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:34.454347 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-bc368e3f-d445-4334-b438-04466ab5b354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:34.455600 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 869/1722] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:43:34 2022] PUT /identity/v3/projects/a03897e0f056406dbb5a586e9dc47d2b/users/74629a2eca3b4266a90d72fe4d43ded6/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:43:34.466071 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69d840-d870-4fca-a5e9-8cd944140f89 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:34.466383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69d840-d870-4fca-a5e9-8cd944140f89 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:34.466671 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69d840-d870-4fca-a5e9-8cd944140f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:34.542412 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-dd69d840-d870-4fca-a5e9-8cd944140f89 None None] MFA Rules not processed for user `74629a2eca3b4266a90d72fe4d43ded6`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:34.543737 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-dd69d840-d870-4fca-a5e9-8cd944140f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:34.591070 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 854/1723] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:34 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:43:34.593333 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc368e3f-d445-4334-b438-04466ab5b354 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:34.596578 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-bc368e3f-d445-4334-b438-04466ab5b354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:34.708740 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 870/1724] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:43:34.715430 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd69d840-d870-4fca-a5e9-8cd944140f89 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:34.722680 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-dd69d840-d870-4fca-a5e9-8cd944140f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:34.782502 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc368e3f-d445-4334-b438-04466ab5b354 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:34.801772 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663be99f-95d2-4826-9e9a-e0e743fb8fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:34.803052 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663be99f-95d2-4826-9e9a-e0e743fb8fa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:34.803572 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663be99f-95d2-4826-9e9a-e0e743fb8fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:34.804179 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663be99f-95d2-4826-9e9a-e0e743fb8fa7 admin admin] PATH_INFO: `/v3/projects/a5a2b3a6de8f4fdc8c7573a3fd3b4bf7` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:34.805961 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663be99f-95d2-4826-9e9a-e0e743fb8fa7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5a2b3a6de8f4fdc8c7573a3fd3b4bf7)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:34.808172 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663be99f-95d2-4826-9e9a-e0e743fb8fa7 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:34.957813 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd69d840-d870-4fca-a5e9-8cd944140f89 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:34.978806 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-663be99f-95d2-4826-9e9a-e0e743fb8fa7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5a2b3a6de8f4fdc8c7573a3fd3b4bf7', 'request_id': 'req-663be99f-95d2-4826-9e9a-e0e743fb8fa7'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:34.991981 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-663be99f-95d2-4826-9e9a-e0e743fb8fa7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5a2b3a6de8f4fdc8c7573a3fd3b4bf7', 'request_id': 'req-663be99f-95d2-4826-9e9a-e0e743fb8fa7'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:35.007891 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:35.026299 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-663be99f-95d2-4826-9e9a-e0e743fb8fa7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5a2b3a6de8f4fdc8c7573a3fd3b4bf7', 'request_id': 'req-663be99f-95d2-4826-9e9a-e0e743fb8fa7'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:35.038199 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:35.060597 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-663be99f-95d2-4826-9e9a-e0e743fb8fa7 admin admin] The token cache is being invalidate because project a5a2b3a6de8f4fdc8c7573a3fd3b4bf7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:43:35.060597 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-663be99f-95d2-4826-9e9a-e0e743fb8fa7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:35.061272 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c9f86-82db-46e4-b1d0-e75c8f1802c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:35.061974 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c9f86-82db-46e4-b1d0-e75c8f1802c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:35.062258 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c9f86-82db-46e4-b1d0-e75c8f1802c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:35.062534 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c9f86-82db-46e4-b1d0-e75c8f1802c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:35.063821 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16c9f86-82db-46e4-b1d0-e75c8f1802c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:35.063922 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-663be99f-95d2-4826-9e9a-e0e743fb8fa7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:35.065806 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16c9f86-82db-46e4-b1d0-e75c8f1802c7 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:35.065926 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 855/1725] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:43:34 2022] DELETE /identity/v3/projects/a5a2b3a6de8f4fdc8c7573a3fd3b4bf7 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:35.069146 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663be99f-95d2-4826-9e9a-e0e743fb8fa7 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:35.071357 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-663be99f-95d2-4826-9e9a-e0e743fb8fa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:35.091869 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663be99f-95d2-4826-9e9a-e0e743fb8fa7 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:35.098332 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791296f-6f9e-423e-bca0-fa8578741ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:35.100037 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791296f-6f9e-423e-bca0-fa8578741ae9 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:35.100986 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791296f-6f9e-423e-bca0-fa8578741ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:35.101746 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791296f-6f9e-423e-bca0-fa8578741ae9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:35.103589 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f791296f-6f9e-423e-bca0-fa8578741ae9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:35.108765 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f791296f-6f9e-423e-bca0-fa8578741ae9 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:35.117073 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 871/1726] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:43:34 2022] POST /identity/v3/projects => generated 405 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:43:35.118187 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c9f86-82db-46e4-b1d0-e75c8f1802c7 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:35.120147 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e16c9f86-82db-46e4-b1d0-e75c8f1802c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:35.169460 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 856/1727] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:43:35 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:43:35.170280 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791296f-6f9e-423e-bca0-fa8578741ae9 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:35.175549 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-f791296f-6f9e-423e-bca0-fa8578741ae9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:35.283016 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c9f86-82db-46e4-b1d0-e75c8f1802c7 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:35.339421 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791296f-6f9e-423e-bca0-fa8578741ae9 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:35.359169 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d800a-f885-4df1-8167-4e3c44710f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:35.359921 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d800a-f885-4df1-8167-4e3c44710f07 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:35.360217 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d800a-f885-4df1-8167-4e3c44710f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:35.360502 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d800a-f885-4df1-8167-4e3c44710f07 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:35.361245 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5d800a-f885-4df1-8167-4e3c44710f07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:35.369829 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5d800a-f885-4df1-8167-4e3c44710f07 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:35.388251 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4199762d-c9a0-437a-8932-d467c7510f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:35.389226 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199762d-c9a0-437a-8932-d467c7510f1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:35.389516 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199762d-c9a0-437a-8932-d467c7510f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:35.389799 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199762d-c9a0-437a-8932-d467c7510f1c admin admin] PATH_INFO: `/v3/users/30409d12ce2040cd9a76736e8569007b` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:35.392355 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4199762d-c9a0-437a-8932-d467c7510f1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=30409d12ce2040cd9a76736e8569007b)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:35.398325 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4199762d-c9a0-437a-8932-d467c7510f1c admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:35.424869 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 857/1728] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:43:35 2022] POST /identity/v3/users => generated 309 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:43:35.426145 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d800a-f885-4df1-8167-4e3c44710f07 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:35.439049 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d800a-f885-4df1-8167-4e3c44710f07 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:35.451235 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917de69-360d-4787-a519-ff74c22bc8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:35.451827 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917de69-360d-4787-a519-ff74c22bc8d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:35.452126 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917de69-360d-4787-a519-ff74c22bc8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:35.452475 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917de69-360d-4787-a519-ff74c22bc8d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:35.453367 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9917de69-360d-4787-a519-ff74c22bc8d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:35.458540 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9917de69-360d-4787-a519-ff74c22bc8d7 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:35.485658 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-4199762d-c9a0-437a-8932-d467c7510f1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30409d12ce2040cd9a76736e8569007b'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:35.486582 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-4199762d-c9a0-437a-8932-d467c7510f1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30409d12ce2040cd9a76736e8569007b'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:35.487297 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-4199762d-c9a0-437a-8932-d467c7510f1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30409d12ce2040cd9a76736e8569007b'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:35.508483 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-4199762d-c9a0-437a-8932-d467c7510f1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30409d12ce2040cd9a76736e8569007b'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:35.509195 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 858/1729] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:43:35 2022] POST /identity/v3/users => generated 299 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:43:35.510377 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917de69-360d-4787-a519-ff74c22bc8d7 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:35.513146 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-9917de69-360d-4787-a519-ff74c22bc8d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:35.531114 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917de69-360d-4787-a519-ff74c22bc8d7 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:35.533516 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-4199762d-c9a0-437a-8932-d467c7510f1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30409d12ce2040cd9a76736e8569007b'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:35.537099 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6592b5-aa52-4f41-9b39-01bc58c1757d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:35.537825 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6592b5-aa52-4f41-9b39-01bc58c1757d admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:35.538121 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6592b5-aa52-4f41-9b39-01bc58c1757d admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:35.539425 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6592b5-aa52-4f41-9b39-01bc58c1757d admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:35.539869 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6592b5-aa52-4f41-9b39-01bc58c1757d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:35.542441 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6592b5-aa52-4f41-9b39-01bc58c1757d admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:35.548437 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 872/1730] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:43:35 2022] DELETE /identity/v3/users/30409d12ce2040cd9a76736e8569007b => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:35.549374 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4199762d-c9a0-437a-8932-d467c7510f1c admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:35.552211 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-4199762d-c9a0-437a-8932-d467c7510f1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:35.561743 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 859/1731] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:43:35.566311 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6592b5-aa52-4f41-9b39-01bc58c1757d admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:35.567633 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:35.579189 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6592b5-aa52-4f41-9b39-01bc58c1757d admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:35.652403 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4199762d-c9a0-437a-8932-d467c7510f1c admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:35.666345 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397c388f-1096-48d7-8ad4-c1d5b87577d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:35.667169 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c388f-1096-48d7-8ad4-c1d5b87577d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:35.667473 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c388f-1096-48d7-8ad4-c1d5b87577d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:35.667888 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c388f-1096-48d7-8ad4-c1d5b87577d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:35.668993 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397c388f-1096-48d7-8ad4-c1d5b87577d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:35.675783 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397c388f-1096-48d7-8ad4-c1d5b87577d6 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:35.681463 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c69228-60e4-47c5-a1ec-31113d5c9cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:35.683188 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c69228-60e4-47c5-a1ec-31113d5c9cd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:35.683732 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c69228-60e4-47c5-a1ec-31113d5c9cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:35.684025 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c69228-60e4-47c5-a1ec-31113d5c9cd9 admin admin] PATH_INFO: `/v3/projects/81a6110a2ddc4d69a76656dd25b565b2/users/3438bf7f1d164791adf02ae7efe03c9c/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:35.695531 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 873/1732] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:43:35.697498 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397c388f-1096-48d7-8ad4-c1d5b87577d6 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:35.707677 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c69228-60e4-47c5-a1ec-31113d5c9cd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81a6110a2ddc4d69a76656dd25b565b2, user_id=3438bf7f1d164791adf02ae7efe03c9c, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:35.713229 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c69228-60e4-47c5-a1ec-31113d5c9cd9 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:35.730922 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397c388f-1096-48d7-8ad4-c1d5b87577d6 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:35.738989 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 860/1733] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:43:35 2022] PUT /identity/v3/projects/81a6110a2ddc4d69a76656dd25b565b2/users/3438bf7f1d164791adf02ae7efe03c9c/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:43:35.749839 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c69228-60e4-47c5-a1ec-31113d5c9cd9 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:35.779225 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=90,keystone:INSERT=9,keystone:DELETE=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:35.916291 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5c7dc5-79a7-4ef4-b366-34a3f900b3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:35.917747 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c7dc5-79a7-4ef4-b366-34a3f900b3c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:35.918240 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c7dc5-79a7-4ef4-b366-34a3f900b3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:35.918760 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c7dc5-79a7-4ef4-b366-34a3f900b3c0 admin admin] PATH_INFO: `/v3/projects/94c779535749491bbda14ec0f6ed0fd6/users/382b1ceca3f9418ca2ea00d16d682ac8/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:35.933874 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c69228-60e4-47c5-a1ec-31113d5c9cd9 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:35.939397 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5c7dc5-79a7-4ef4-b366-34a3f900b3c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94c779535749491bbda14ec0f6ed0fd6, user_id=382b1ceca3f9418ca2ea00d16d682ac8, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:35.941658 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac77e4-3e4d-408c-b4d1-cdf35e2af6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:35.942615 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac77e4-3e4d-408c-b4d1-cdf35e2af6e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:35.942990 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5c7dc5-79a7-4ef4-b366-34a3f900b3c0 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:35.943093 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac77e4-3e4d-408c-b4d1-cdf35e2af6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:35.943456 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac77e4-3e4d-408c-b4d1-cdf35e2af6e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:35.944753 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abac77e4-3e4d-408c-b4d1-cdf35e2af6e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:35.947772 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abac77e4-3e4d-408c-b4d1-cdf35e2af6e9 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:35.966306 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 874/1734] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:43:35 2022] PUT /identity/v3/projects/94c779535749491bbda14ec0f6ed0fd6/users/382b1ceca3f9418ca2ea00d16d682ac8/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:43:35.971919 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 861/1735] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:43:35.972061 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5c7dc5-79a7-4ef4-b366-34a3f900b3c0 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:35.974647 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac77e4-3e4d-408c-b4d1-cdf35e2af6e9 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:35.996631 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac77e4-3e4d-408c-b4d1-cdf35e2af6e9 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:36.078027 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5c7dc5-79a7-4ef4-b366-34a3f900b3c0 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:36.088712 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d086fe-d767-414c-b8f0-8df58d591929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:36.089458 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d086fe-d767-414c-b8f0-8df58d591929 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:36.089743 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d086fe-d767-414c-b8f0-8df58d591929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:36.090017 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d086fe-d767-414c-b8f0-8df58d591929 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:36.090759 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d086fe-d767-414c-b8f0-8df58d591929 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:36.092423 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d086fe-d767-414c-b8f0-8df58d591929 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:36.103731 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cc1ccf-c23d-4568-a06e-a3b605ca7db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:36.104644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cc1ccf-c23d-4568-a06e-a3b605ca7db0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:36.104929 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cc1ccf-c23d-4568-a06e-a3b605ca7db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:36.105240 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cc1ccf-c23d-4568-a06e-a3b605ca7db0 admin admin] PATH_INFO: `/v3/projects/81a6110a2ddc4d69a76656dd25b565b2/users/3438bf7f1d164791adf02ae7efe03c9c/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:36.119807 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cc1ccf-c23d-4568-a06e-a3b605ca7db0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81a6110a2ddc4d69a76656dd25b565b2, user_id=3438bf7f1d164791adf02ae7efe03c9c, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:36.119807 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cc1ccf-c23d-4568-a06e-a3b605ca7db0 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:36.119989 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 875/1736] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:43:36.119989 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d086fe-d767-414c-b8f0-8df58d591929 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:36.125094 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-85d086fe-d767-414c-b8f0-8df58d591929 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:36.137702 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 862/1737] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:43:35 2022] PUT /identity/v3/projects/81a6110a2ddc4d69a76656dd25b565b2/users/3438bf7f1d164791adf02ae7efe03c9c/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:43:36.138825 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cc1ccf-c23d-4568-a06e-a3b605ca7db0 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:36.145934 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d086fe-d767-414c-b8f0-8df58d591929 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:36.173338 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:36.239718 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cc1ccf-c23d-4568-a06e-a3b605ca7db0 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:36.242006 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26b37c0-8a17-4bb3-a277-b2f8d56017d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:36.242889 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b37c0-8a17-4bb3-a277-b2f8d56017d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:36.243179 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b37c0-8a17-4bb3-a277-b2f8d56017d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:36.243466 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b37c0-8a17-4bb3-a277-b2f8d56017d5 admin admin] PATH_INFO: `/v3/projects/776ce9b37bfb44909f448d6eea08599a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:36.244692 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26b37c0-8a17-4bb3-a277-b2f8d56017d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=776ce9b37bfb44909f448d6eea08599a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:36.245597 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2013f9e5-942a-42a9-b27c-7c1f07cc7074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:36.246653 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2013f9e5-942a-42a9-b27c-7c1f07cc7074 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:36.246973 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26b37c0-8a17-4bb3-a277-b2f8d56017d5 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:36.247298 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2013f9e5-942a-42a9-b27c-7c1f07cc7074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:36.247704 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2013f9e5-942a-42a9-b27c-7c1f07cc7074 admin admin] PATH_INFO: `/v3/projects/94c779535749491bbda14ec0f6ed0fd6/users/382b1ceca3f9418ca2ea00d16d682ac8/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:36.250474 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2013f9e5-942a-42a9-b27c-7c1f07cc7074 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94c779535749491bbda14ec0f6ed0fd6, user_id=382b1ceca3f9418ca2ea00d16d682ac8, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:36.253585 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2013f9e5-942a-42a9-b27c-7c1f07cc7074 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:36.266460 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 863/1738] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:43:36 2022] PUT /identity/v3/projects/94c779535749491bbda14ec0f6ed0fd6/users/382b1ceca3f9418ca2ea00d16d682ac8/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:36.278475 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646416c7-6bf1-498e-9123-b0eb9b272bbd None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:36.278826 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646416c7-6bf1-498e-9123-b0eb9b272bbd None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:36.279172 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646416c7-6bf1-498e-9123-b0eb9b272bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:36.330613 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-646416c7-6bf1-498e-9123-b0eb9b272bbd None None] MFA Rules not processed for user `382b1ceca3f9418ca2ea00d16d682ac8`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:36.331849 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-646416c7-6bf1-498e-9123-b0eb9b272bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:36.346292 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-f26b37c0-8a17-4bb3-a277-b2f8d56017d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '776ce9b37bfb44909f448d6eea08599a', 'request_id': 'req-f26b37c0-8a17-4bb3-a277-b2f8d56017d5'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:36.359039 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-f26b37c0-8a17-4bb3-a277-b2f8d56017d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '776ce9b37bfb44909f448d6eea08599a', 'request_id': 'req-f26b37c0-8a17-4bb3-a277-b2f8d56017d5'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:36.373713 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-f26b37c0-8a17-4bb3-a277-b2f8d56017d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '776ce9b37bfb44909f448d6eea08599a', 'request_id': 'req-f26b37c0-8a17-4bb3-a277-b2f8d56017d5'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:36.379809 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:36.404103 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-f26b37c0-8a17-4bb3-a277-b2f8d56017d5 admin admin] The token cache is being invalidate because project 776ce9b37bfb44909f448d6eea08599a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:43:36.404103 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-f26b37c0-8a17-4bb3-a277-b2f8d56017d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:36.404103 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-f26b37c0-8a17-4bb3-a277-b2f8d56017d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:36.404103 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 876/1739] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:43:36 2022] DELETE /identity/v3/projects/776ce9b37bfb44909f448d6eea08599a => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:36.409408 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82de028-98e5-4895-ae5f-f41327624086 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:36.409872 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82de028-98e5-4895-ae5f-f41327624086 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:36.410639 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82de028-98e5-4895-ae5f-f41327624086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:36.446934 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-e82de028-98e5-4895-ae5f-f41327624086 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:36.448079 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e82de028-98e5-4895-ae5f-f41327624086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:36.512471 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 864/1740] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:36 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:36.524161 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 877/1741] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:36.533351 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88bce71-c5b5-4708-84f3-acbcf868ecce None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:36.533651 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88bce71-c5b5-4708-84f3-acbcf868ecce None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:36.533939 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88bce71-c5b5-4708-84f3-acbcf868ecce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:36.541134 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82de028-98e5-4895-ae5f-f41327624086 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:36.546609 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e82de028-98e5-4895-ae5f-f41327624086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:36.574152 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-d88bce71-c5b5-4708-84f3-acbcf868ecce None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:36.575348 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-d88bce71-c5b5-4708-84f3-acbcf868ecce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:36.591306 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 865/1742] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:36.596464 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88bce71-c5b5-4708-84f3-acbcf868ecce None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:36.607818 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88bce71-c5b5-4708-84f3-acbcf868ecce None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:36.614163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fb9156-75ea-4339-9267-90cd243c8b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:36.614848 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb9156-75ea-4339-9267-90cd243c8b8b admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:36.615132 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb9156-75ea-4339-9267-90cd243c8b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:36.615410 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb9156-75ea-4339-9267-90cd243c8b8b admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:36.616247 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fb9156-75ea-4339-9267-90cd243c8b8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:43:36.616839 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fb9156-75ea-4339-9267-90cd243c8b8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:36.618391 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fb9156-75ea-4339-9267-90cd243c8b8b admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:36.638111 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 866/1743] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:43:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:43:36.639137 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fb9156-75ea-4339-9267-90cd243c8b8b admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:36.650682 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fb9156-75ea-4339-9267-90cd243c8b8b admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:36.661735 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3084b0be-a999-4825-af89-bfdf1c85616e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:36.662612 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3084b0be-a999-4825-af89-bfdf1c85616e admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:36.662906 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3084b0be-a999-4825-af89-bfdf1c85616e admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:36.663362 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3084b0be-a999-4825-af89-bfdf1c85616e admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:36.664140 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3084b0be-a999-4825-af89-bfdf1c85616e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:36.666281 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3084b0be-a999-4825-af89-bfdf1c85616e admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:36.678004 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82de028-98e5-4895-ae5f-f41327624086 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:36.684994 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c662eee2-4376-4e20-8268-70addda2c5a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:36.685854 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c662eee2-4376-4e20-8268-70addda2c5a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:36.686194 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c662eee2-4376-4e20-8268-70addda2c5a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:36.686585 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c662eee2-4376-4e20-8268-70addda2c5a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:36.695984 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:36.699582 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c662eee2-4376-4e20-8268-70addda2c5a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:36.702243 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c662eee2-4376-4e20-8268-70addda2c5a7 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:36.705432 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 867/1744] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:43:36 2022] POST /identity/v3/projects => generated 387 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:43:36.707481 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3084b0be-a999-4825-af89-bfdf1c85616e admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:36.721639 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 878/1745] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:36 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:43:36.831484 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3084b0be-a999-4825-af89-bfdf1c85616e admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:36.839036 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ae352-f624-41dd-bd27-c9f4f7f573d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:36.839736 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ae352-f624-41dd-bd27-c9f4f7f573d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:36.840022 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ae352-f624-41dd-bd27-c9f4f7f573d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:36.840303 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ae352-f624-41dd-bd27-c9f4f7f573d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:36.841019 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ae352-f624-41dd-bd27-c9f4f7f573d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:36.843063 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ae352-f624-41dd-bd27-c9f4f7f573d2 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:36.869311 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 868/1746] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:43:36 2022] POST /identity/v3/users => generated 294 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:43:36.873655 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c662eee2-4376-4e20-8268-70addda2c5a7 service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:36.892685 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c662eee2-4376-4e20-8268-70addda2c5a7 service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:36.901826 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d4139d-b5a8-47ac-b087-171157024e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:36.903778 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4139d-b5a8-47ac-b087-171157024e5a admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:36.904164 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4139d-b5a8-47ac-b087-171157024e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:36.904553 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4139d-b5a8-47ac-b087-171157024e5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:36.905599 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d4139d-b5a8-47ac-b087-171157024e5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:36.908044 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d4139d-b5a8-47ac-b087-171157024e5a admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:36.951015 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 879/1747] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:36 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:43:36.960623 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ae352-f624-41dd-bd27-c9f4f7f573d2 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:36.982265 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ae352-f624-41dd-bd27-c9f4f7f573d2 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:36.991829 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8e22d1-d194-4119-a277-2185b33c1947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:36.992620 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e22d1-d194-4119-a277-2185b33c1947 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:36.992919 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e22d1-d194-4119-a277-2185b33c1947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:36.993293 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e22d1-d194-4119-a277-2185b33c1947 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:36.994113 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8e22d1-d194-4119-a277-2185b33c1947 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:36.996971 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8e22d1-d194-4119-a277-2185b33c1947 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:37.011713 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 869/1748] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:36 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:43:37.015553 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d4139d-b5a8-47ac-b087-171157024e5a admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:37.030845 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d4139d-b5a8-47ac-b087-171157024e5a admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:37.043597 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac45e84c-394e-4f8e-9d5d-38ee385c3f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:37.044780 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45e84c-394e-4f8e-9d5d-38ee385c3f44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:37.045346 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45e84c-394e-4f8e-9d5d-38ee385c3f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:37.046940 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45e84c-394e-4f8e-9d5d-38ee385c3f44 admin admin] PATH_INFO: `/v3/projects/b32e568b74df41e5ba5709d076198ffa/users/b7a6ab5870dc4865865ea55f5c3b2ca5/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:37.070517 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac45e84c-394e-4f8e-9d5d-38ee385c3f44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b32e568b74df41e5ba5709d076198ffa, user_id=b7a6ab5870dc4865865ea55f5c3b2ca5, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:37.073405 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac45e84c-394e-4f8e-9d5d-38ee385c3f44 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:37.091260 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 880/1749] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:43:37 2022] PUT /identity/v3/projects/b32e568b74df41e5ba5709d076198ffa/users/b7a6ab5870dc4865865ea55f5c3b2ca5/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:43:37.102186 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b68e9b-b4a8-45b4-804c-690d2fd7e8d5 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:37.102519 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b68e9b-b4a8-45b4-804c-690d2fd7e8d5 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:37.102977 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b68e9b-b4a8-45b4-804c-690d2fd7e8d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:37.142446 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-a4b68e9b-b4a8-45b4-804c-690d2fd7e8d5 None None] MFA Rules not processed for user `b7a6ab5870dc4865865ea55f5c3b2ca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:37.143674 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-a4b68e9b-b4a8-45b4-804c-690d2fd7e8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:37.271813 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:37.329638 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 870/1750] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:37.335122 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac45e84c-394e-4f8e-9d5d-38ee385c3f44 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:37.421096 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac45e84c-394e-4f8e-9d5d-38ee385c3f44 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:37.428828 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc6052a-9c01-4125-976f-39a82b7b1107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:37.429560 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6052a-9c01-4125-976f-39a82b7b1107 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:37.429843 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6052a-9c01-4125-976f-39a82b7b1107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:37.430127 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6052a-9c01-4125-976f-39a82b7b1107 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:37.430829 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc6052a-9c01-4125-976f-39a82b7b1107 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:37.432763 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc6052a-9c01-4125-976f-39a82b7b1107 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:37.470316 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 881/1751] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:43:37 2022] POST /identity/v3/projects => generated 385 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:43:37.474450 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b68e9b-b4a8-45b4-804c-690d2fd7e8d5 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:37.488806 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b68e9b-b4a8-45b4-804c-690d2fd7e8d5 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:37.580626 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea61c1-ebea-4b24-a380-06fa4fb410c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:37.581826 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea61c1-ebea-4b24-a380-06fa4fb410c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:37.582700 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea61c1-ebea-4b24-a380-06fa4fb410c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:37.583130 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea61c1-ebea-4b24-a380-06fa4fb410c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:37.584384 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beea61c1-ebea-4b24-a380-06fa4fb410c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:37.587800 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beea61c1-ebea-4b24-a380-06fa4fb410c0 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:37.602354 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb6087-a9cd-4e93-8f93-97cda9ebb8ac None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:37.602681 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb6087-a9cd-4e93-8f93-97cda9ebb8ac None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:37.602976 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb6087-a9cd-4e93-8f93-97cda9ebb8ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:37.613788 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 871/1752] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:43:37 2022] POST /identity/v3/users => generated 299 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:43:37.617724 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea61c1-ebea-4b24-a380-06fa4fb410c0 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:37.644218 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea61c1-ebea-4b24-a380-06fa4fb410c0 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:37.651215 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-6acb6087-a9cd-4e93-8f93-97cda9ebb8ac None None] MFA Rules not processed for user `3438bf7f1d164791adf02ae7efe03c9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:37.652811 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-6acb6087-a9cd-4e93-8f93-97cda9ebb8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:37.659423 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab77317c-538c-4043-bae5-7198f3238a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:37.663080 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77317c-538c-4043-bae5-7198f3238a94 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:37.663080 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77317c-538c-4043-bae5-7198f3238a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:37.663080 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77317c-538c-4043-bae5-7198f3238a94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:37.663080 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab77317c-538c-4043-bae5-7198f3238a94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:37.671423 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab77317c-538c-4043-bae5-7198f3238a94 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:37.692112 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 872/1753] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:37 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:43:37.692849 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab77317c-538c-4043-bae5-7198f3238a94 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:37.696349 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ab77317c-538c-4043-bae5-7198f3238a94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:37.720451 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab77317c-538c-4043-bae5-7198f3238a94 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:37.727783 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e050c0-bb46-4f92-8487-0720f7f9ec95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:37.728732 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e050c0-bb46-4f92-8487-0720f7f9ec95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:37.729008 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e050c0-bb46-4f92-8487-0720f7f9ec95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:37.729335 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e050c0-bb46-4f92-8487-0720f7f9ec95 admin admin] PATH_INFO: `/v3/users/74629a2eca3b4266a90d72fe4d43ded6` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:37.730639 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e050c0-bb46-4f92-8487-0720f7f9ec95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74629a2eca3b4266a90d72fe4d43ded6)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:37.733117 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e050c0-bb46-4f92-8487-0720f7f9ec95 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:37.776361 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 882/1754] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:37 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:43:37.777567 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acb6087-a9cd-4e93-8f93-97cda9ebb8ac None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:37.793979 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acb6087-a9cd-4e93-8f93-97cda9ebb8ac None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:37.798768 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77014b9e-fc94-4fea-a14f-296a5ca87b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:37.799793 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77014b9e-fc94-4fea-a14f-296a5ca87b47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:37.800101 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77014b9e-fc94-4fea-a14f-296a5ca87b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:37.800581 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77014b9e-fc94-4fea-a14f-296a5ca87b47 admin admin] PATH_INFO: `/v3/projects/b0a865a35946438d825a8bb09ed83944/users/3adce294c88e4db1a08b327294d54caa/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:37.823121 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77014b9e-fc94-4fea-a14f-296a5ca87b47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0a865a35946438d825a8bb09ed83944, user_id=3adce294c88e4db1a08b327294d54caa, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:37.826487 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77014b9e-fc94-4fea-a14f-296a5ca87b47 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:37.831450 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b8e050c0-bb46-4f92-8487-0720f7f9ec95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74629a2eca3b4266a90d72fe4d43ded6'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:37.832194 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b8e050c0-bb46-4f92-8487-0720f7f9ec95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74629a2eca3b4266a90d72fe4d43ded6'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:37.840157 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b8e050c0-bb46-4f92-8487-0720f7f9ec95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74629a2eca3b4266a90d72fe4d43ded6'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:37.845896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 883/1755] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:43:37 2022] PUT /identity/v3/projects/b0a865a35946438d825a8bb09ed83944/users/3adce294c88e4db1a08b327294d54caa/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:37.849653 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77014b9e-fc94-4fea-a14f-296a5ca87b47 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:37.858095 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b8e050c0-bb46-4f92-8487-0720f7f9ec95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74629a2eca3b4266a90d72fe4d43ded6'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:37.858460 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:37.859251 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b8e050c0-bb46-4f92-8487-0720f7f9ec95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74629a2eca3b4266a90d72fe4d43ded6'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:37.875165 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 873/1756] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:43:37 2022] DELETE /identity/v3/users/74629a2eca3b4266a90d72fe4d43ded6 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:37.879957 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65be125-1717-442c-bf4f-2e5e04c51fa6 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:37.883506 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65be125-1717-442c-bf4f-2e5e04c51fa6 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:37.883634 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65be125-1717-442c-bf4f-2e5e04c51fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:37.913122 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-b65be125-1717-442c-bf4f-2e5e04c51fa6 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:37.914621 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-b65be125-1717-442c-bf4f-2e5e04c51fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:37.945685 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77014b9e-fc94-4fea-a14f-296a5ca87b47 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:37.969767 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:38.013454 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 874/1757] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:43:38.021646 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41fa903-cc3e-4aa2-a24b-715f85894e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:38.022337 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41fa903-cc3e-4aa2-a24b-715f85894e3d admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:38.022628 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41fa903-cc3e-4aa2-a24b-715f85894e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:38.023057 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41fa903-cc3e-4aa2-a24b-715f85894e3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:38.023903 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41fa903-cc3e-4aa2-a24b-715f85894e3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:38.024889 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65be125-1717-442c-bf4f-2e5e04c51fa6 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:38.025558 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41fa903-cc3e-4aa2-a24b-715f85894e3d admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:38.026909 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-b65be125-1717-442c-bf4f-2e5e04c51fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:38.036777 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 884/1758] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:37 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:43:38.041193 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41fa903-cc3e-4aa2-a24b-715f85894e3d admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:38.043092 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b41fa903-cc3e-4aa2-a24b-715f85894e3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:38.084320 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41fa903-cc3e-4aa2-a24b-715f85894e3d admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:38.096272 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b122b-3256-4940-8dbc-63831eb4bec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:38.097223 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b122b-3256-4940-8dbc-63831eb4bec0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:38.097571 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b122b-3256-4940-8dbc-63831eb4bec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:38.097864 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b122b-3256-4940-8dbc-63831eb4bec0 admin admin] PATH_INFO: `/v3/projects/b0a865a35946438d825a8bb09ed83944/users/3adce294c88e4db1a08b327294d54caa/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:38.100052 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b122b-3256-4940-8dbc-63831eb4bec0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0a865a35946438d825a8bb09ed83944, user_id=3adce294c88e4db1a08b327294d54caa, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:38.102931 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b122b-3256-4940-8dbc-63831eb4bec0 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:38.113468 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65be125-1717-442c-bf4f-2e5e04c51fa6 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:38.119327 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 885/1759] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:43:38 2022] PUT /identity/v3/projects/b0a865a35946438d825a8bb09ed83944/users/3adce294c88e4db1a08b327294d54caa/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:43:38.131856 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03337e35-4cb6-4caa-aacf-9ca3505599ed None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:38.132586 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03337e35-4cb6-4caa-aacf-9ca3505599ed None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:38.132884 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03337e35-4cb6-4caa-aacf-9ca3505599ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:38.177041 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-03337e35-4cb6-4caa-aacf-9ca3505599ed None None] MFA Rules not processed for user `3adce294c88e4db1a08b327294d54caa`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:38.178835 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-03337e35-4cb6-4caa-aacf-9ca3505599ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:38.197966 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6798f83d-25e9-4386-ba1e-3c9b7c833833 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:38.198685 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6798f83d-25e9-4386-ba1e-3c9b7c833833 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:38.198976 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6798f83d-25e9-4386-ba1e-3c9b7c833833 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:38.199262 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6798f83d-25e9-4386-ba1e-3c9b7c833833 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:38.279973 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 886/1760] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:38 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:38.294327 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03337e35-4cb6-4caa-aacf-9ca3505599ed None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:38.296776 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-03337e35-4cb6-4caa-aacf-9ca3505599ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:38.351776 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6798f83d-25e9-4386-ba1e-3c9b7c833833 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:38.356207 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6798f83d-25e9-4386-ba1e-3c9b7c833833 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:38.375323 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:38.376824 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 875/1761] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:43:38.440336 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03337e35-4cb6-4caa-aacf-9ca3505599ed None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:38.444816 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f70b7a-6bdb-4145-8eee-bdb8c409a77b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:38.445517 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f70b7a-6bdb-4145-8eee-bdb8c409a77b service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:38.445925 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f70b7a-6bdb-4145-8eee-bdb8c409a77b service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:38.446123 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f70b7a-6bdb-4145-8eee-bdb8c409a77b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:38.456083 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f70b7a-6bdb-4145-8eee-bdb8c409a77b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:38.457734 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f70b7a-6bdb-4145-8eee-bdb8c409a77b service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:38.468297 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 887/1762] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:38 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:43:38.497850 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6798f83d-25e9-4386-ba1e-3c9b7c833833 service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:38.509564 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6798f83d-25e9-4386-ba1e-3c9b7c833833 service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:38.517793 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4453f-8338-43a8-8c35-6c97d5e04bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:38.518708 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4453f-8338-43a8-8c35-6c97d5e04bf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:38.518995 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4453f-8338-43a8-8c35-6c97d5e04bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:38.519271 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4453f-8338-43a8-8c35-6c97d5e04bf5 admin admin] PATH_INFO: `/v3/users/b7a6ab5870dc4865865ea55f5c3b2ca5` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:38.520567 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c4453f-8338-43a8-8c35-6c97d5e04bf5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7a6ab5870dc4865865ea55f5c3b2ca5)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:38.522793 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c4453f-8338-43a8-8c35-6c97d5e04bf5 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:38.572162 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:38.584534 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f8c4453f-8338-43a8-8c35-6c97d5e04bf5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7a6ab5870dc4865865ea55f5c3b2ca5'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:38.585697 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f8c4453f-8338-43a8-8c35-6c97d5e04bf5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7a6ab5870dc4865865ea55f5c3b2ca5'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:38.595560 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f8c4453f-8338-43a8-8c35-6c97d5e04bf5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7a6ab5870dc4865865ea55f5c3b2ca5'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:38.611277 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f8c4453f-8338-43a8-8c35-6c97d5e04bf5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7a6ab5870dc4865865ea55f5c3b2ca5'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:38.612045 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f8c4453f-8338-43a8-8c35-6c97d5e04bf5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7a6ab5870dc4865865ea55f5c3b2ca5'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:38.625765 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 876/1763] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:43:38 2022] DELETE /identity/v3/users/b7a6ab5870dc4865865ea55f5c3b2ca5 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:38.630348 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b208a-7ae7-460a-a458-c14438193970 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:38.630645 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b208a-7ae7-460a-a458-c14438193970 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:38.630957 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b208a-7ae7-460a-a458-c14438193970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:38.648927 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-312b208a-7ae7-460a-a458-c14438193970 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:38.650246 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-312b208a-7ae7-460a-a458-c14438193970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:38.713426 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 888/1764] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:38.724430 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4453f-8338-43a8-8c35-6c97d5e04bf5 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:38.725946 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-f8c4453f-8338-43a8-8c35-6c97d5e04bf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:38.787411 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4453f-8338-43a8-8c35-6c97d5e04bf5 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:38.791286 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2f14a-96a5-45bb-aaf2-f2a02cfad91c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:38.791998 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2f14a-96a5-45bb-aaf2-f2a02cfad91c service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:38.792324 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2f14a-96a5-45bb-aaf2-f2a02cfad91c service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:38.792619 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2f14a-96a5-45bb-aaf2-f2a02cfad91c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:38.803860 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d2f14a-96a5-45bb-aaf2-f2a02cfad91c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:38.805575 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d2f14a-96a5-45bb-aaf2-f2a02cfad91c service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:38.817663 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 877/1765] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:43:39.233000 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312b208a-7ae7-460a-a458-c14438193970 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:39.235313 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-312b208a-7ae7-460a-a458-c14438193970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:39.258617 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312b208a-7ae7-460a-a458-c14438193970 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:39.266980 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78ebe9-f978-4d36-9626-3617e4835dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:39.268281 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78ebe9-f978-4d36-9626-3617e4835dc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:39.268574 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78ebe9-f978-4d36-9626-3617e4835dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:39.268924 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78ebe9-f978-4d36-9626-3617e4835dc1 admin admin] PATH_INFO: `/v3/projects/b32e568b74df41e5ba5709d076198ffa` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:39.270330 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a78ebe9-f978-4d36-9626-3617e4835dc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b32e568b74df41e5ba5709d076198ffa)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:39.272458 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a78ebe9-f978-4d36-9626-3617e4835dc1 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:39.327140 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-7a78ebe9-f978-4d36-9626-3617e4835dc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b32e568b74df41e5ba5709d076198ffa', 'request_id': 'req-7a78ebe9-f978-4d36-9626-3617e4835dc1'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:39.334018 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-7a78ebe9-f978-4d36-9626-3617e4835dc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b32e568b74df41e5ba5709d076198ffa', 'request_id': 'req-7a78ebe9-f978-4d36-9626-3617e4835dc1'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:39.346925 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-7a78ebe9-f978-4d36-9626-3617e4835dc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b32e568b74df41e5ba5709d076198ffa', 'request_id': 'req-7a78ebe9-f978-4d36-9626-3617e4835dc1'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:39.351499 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2f14a-96a5-45bb-aaf2-f2a02cfad91c service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:39.356978 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-e4d2f14a-96a5-45bb-aaf2-f2a02cfad91c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:39.372069 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2f14a-96a5-45bb-aaf2-f2a02cfad91c service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:39.389945 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-7a78ebe9-f978-4d36-9626-3617e4835dc1 admin admin] The token cache is being invalidate because project b32e568b74df41e5ba5709d076198ffa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:43:39.389945 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-7a78ebe9-f978-4d36-9626-3617e4835dc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:39.392400 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-7a78ebe9-f978-4d36-9626-3617e4835dc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:39.394832 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 889/1766] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:43:39 2022] DELETE /identity/v3/projects/b32e568b74df41e5ba5709d076198ffa => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:39.399860 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78ebe9-f978-4d36-9626-3617e4835dc1 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:39.401951 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-7a78ebe9-f978-4d36-9626-3617e4835dc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:39.448984 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7079c6-b075-44b0-9872-302d67a48748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:39.449312 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7079c6-b075-44b0-9872-302d67a48748 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:39.449606 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7079c6-b075-44b0-9872-302d67a48748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:39.449888 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7079c6-b075-44b0-9872-302d67a48748 admin admin] PATH_INFO: `/v3/projects/a03897e0f056406dbb5a586e9dc47d2b` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:39.451319 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7079c6-b075-44b0-9872-302d67a48748 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a03897e0f056406dbb5a586e9dc47d2b)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:39.453203 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7079c6-b075-44b0-9872-302d67a48748 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:39.476661 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:39.521893 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-be7079c6-b075-44b0-9872-302d67a48748 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a03897e0f056406dbb5a586e9dc47d2b', 'request_id': 'req-be7079c6-b075-44b0-9872-302d67a48748'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:39.526123 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78ebe9-f978-4d36-9626-3617e4835dc1 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:39.531883 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-be7079c6-b075-44b0-9872-302d67a48748 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a03897e0f056406dbb5a586e9dc47d2b', 'request_id': 'req-be7079c6-b075-44b0-9872-302d67a48748'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:39.533860 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:39.550676 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-be7079c6-b075-44b0-9872-302d67a48748 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a03897e0f056406dbb5a586e9dc47d2b', 'request_id': 'req-be7079c6-b075-44b0-9872-302d67a48748'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:39.572030 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-be7079c6-b075-44b0-9872-302d67a48748 admin admin] The token cache is being invalidate because project a03897e0f056406dbb5a586e9dc47d2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:43:39.572341 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-be7079c6-b075-44b0-9872-302d67a48748 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:39.573094 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-be7079c6-b075-44b0-9872-302d67a48748 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:39.575298 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 878/1767] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:43:39 2022] DELETE /identity/v3/projects/a03897e0f056406dbb5a586e9dc47d2b => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:39.580523 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7079c6-b075-44b0-9872-302d67a48748 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:39.583317 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-be7079c6-b075-44b0-9872-302d67a48748 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:39.640041 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee77d3e4-4210-4df9-9495-92493f5341ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:39.642146 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee77d3e4-4210-4df9-9495-92493f5341ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:39.642146 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee77d3e4-4210-4df9-9495-92493f5341ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:39.642402 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee77d3e4-4210-4df9-9495-92493f5341ba admin admin] PATH_INFO: `/v3/users/3adce294c88e4db1a08b327294d54caa` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:39.644148 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee77d3e4-4210-4df9-9495-92493f5341ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=3adce294c88e4db1a08b327294d54caa)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:39.646312 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee77d3e4-4210-4df9-9495-92493f5341ba admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:39.704548 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7079c6-b075-44b0-9872-302d67a48748 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:39.716422 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f586a23b-2918-4c2d-a483-481526eab5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:39.717943 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586a23b-2918-4c2d-a483-481526eab5ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:39.718550 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586a23b-2918-4c2d-a483-481526eab5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:39.719124 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586a23b-2918-4c2d-a483-481526eab5ab admin admin] PATH_INFO: `/v3/users/382b1ceca3f9418ca2ea00d16d682ac8` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:39.721433 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f586a23b-2918-4c2d-a483-481526eab5ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=382b1ceca3f9418ca2ea00d16d682ac8)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:39.724737 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f586a23b-2918-4c2d-a483-481526eab5ab admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:39.735468 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ee77d3e4-4210-4df9-9495-92493f5341ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3adce294c88e4db1a08b327294d54caa'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:39.748202 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ee77d3e4-4210-4df9-9495-92493f5341ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3adce294c88e4db1a08b327294d54caa'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:39.748202 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ee77d3e4-4210-4df9-9495-92493f5341ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3adce294c88e4db1a08b327294d54caa'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:39.755645 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ee77d3e4-4210-4df9-9495-92493f5341ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3adce294c88e4db1a08b327294d54caa'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:39.782926 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ee77d3e4-4210-4df9-9495-92493f5341ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3adce294c88e4db1a08b327294d54caa'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:39.794241 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f586a23b-2918-4c2d-a483-481526eab5ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '382b1ceca3f9418ca2ea00d16d682ac8'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:39.795060 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f586a23b-2918-4c2d-a483-481526eab5ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '382b1ceca3f9418ca2ea00d16d682ac8'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:39.805197 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 890/1768] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:43:39 2022] DELETE /identity/v3/users/3adce294c88e4db1a08b327294d54caa => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:39.805358 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f586a23b-2918-4c2d-a483-481526eab5ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '382b1ceca3f9418ca2ea00d16d682ac8'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:39.820778 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f586a23b-2918-4c2d-a483-481526eab5ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '382b1ceca3f9418ca2ea00d16d682ac8'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:39.821951 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f586a23b-2918-4c2d-a483-481526eab5ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '382b1ceca3f9418ca2ea00d16d682ac8'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:39.838130 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 879/1769] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:43:39 2022] DELETE /identity/v3/users/382b1ceca3f9418ca2ea00d16d682ac8 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:39.871545 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee77d3e4-4210-4df9-9495-92493f5341ba admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:39.873031 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-ee77d3e4-4210-4df9-9495-92493f5341ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:39.954034 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee77d3e4-4210-4df9-9495-92493f5341ba admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:39.973375 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:40.017476 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e95858-728a-409b-a7d9-992b247228a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:40.018368 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e95858-728a-409b-a7d9-992b247228a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:40.018677 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e95858-728a-409b-a7d9-992b247228a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:40.018931 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e95858-728a-409b-a7d9-992b247228a8 admin admin] PATH_INFO: `/v3/users/0e8043294f8e44ec8d640d64abe44004` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:40.020142 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e95858-728a-409b-a7d9-992b247228a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e8043294f8e44ec8d640d64abe44004)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:40.022130 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e95858-728a-409b-a7d9-992b247228a8 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:40.114420 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-92e95858-728a-409b-a7d9-992b247228a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e8043294f8e44ec8d640d64abe44004'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:40.115144 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-92e95858-728a-409b-a7d9-992b247228a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e8043294f8e44ec8d640d64abe44004'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:40.115851 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-92e95858-728a-409b-a7d9-992b247228a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e8043294f8e44ec8d640d64abe44004'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:40.133569 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-92e95858-728a-409b-a7d9-992b247228a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e8043294f8e44ec8d640d64abe44004'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:40.158863 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-92e95858-728a-409b-a7d9-992b247228a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e8043294f8e44ec8d640d64abe44004'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:40.190618 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 891/1770] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:43:39 2022] DELETE /identity/v3/users/0e8043294f8e44ec8d640d64abe44004 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:40.689631 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f586a23b-2918-4c2d-a483-481526eab5ab admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:40.694653 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-f586a23b-2918-4c2d-a483-481526eab5ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:40.722178 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f586a23b-2918-4c2d-a483-481526eab5ab admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:40.812403 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e95858-728a-409b-a7d9-992b247228a8 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:40.814742 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-92e95858-728a-409b-a7d9-992b247228a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:40.880898 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d492dac-fc89-49d9-8777-f9977267ab3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:40.883293 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d492dac-fc89-49d9-8777-f9977267ab3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:40.883535 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d492dac-fc89-49d9-8777-f9977267ab3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:40.883755 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d492dac-fc89-49d9-8777-f9977267ab3a admin admin] PATH_INFO: `/v3/users/4a8961c2ccef4a5c9d73560037a23a60` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:40.885138 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d492dac-fc89-49d9-8777-f9977267ab3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a8961c2ccef4a5c9d73560037a23a60)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:40.892135 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d492dac-fc89-49d9-8777-f9977267ab3a admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:40.905043 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:40.945375 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e95858-728a-409b-a7d9-992b247228a8 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:40.967254 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ac31f-2c2c-4126-8260-30ecdaea9298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:40.968379 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ac31f-2c2c-4126-8260-30ecdaea9298 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:40.968679 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ac31f-2c2c-4126-8260-30ecdaea9298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:40.969724 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ac31f-2c2c-4126-8260-30ecdaea9298 admin admin] PATH_INFO: `/v3/projects/b0a865a35946438d825a8bb09ed83944` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:40.973999 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8ac31f-2c2c-4126-8260-30ecdaea9298 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0a865a35946438d825a8bb09ed83944)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:40.976016 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8ac31f-2c2c-4126-8260-30ecdaea9298 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:41.003176 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-1d492dac-fc89-49d9-8777-f9977267ab3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a8961c2ccef4a5c9d73560037a23a60'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:41.004519 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-1d492dac-fc89-49d9-8777-f9977267ab3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a8961c2ccef4a5c9d73560037a23a60'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:41.029764 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-1d492dac-fc89-49d9-8777-f9977267ab3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a8961c2ccef4a5c9d73560037a23a60'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:41.041864 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:41.046123 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ad8ac31f-2c2c-4126-8260-30ecdaea9298 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0a865a35946438d825a8bb09ed83944', 'request_id': 'req-ad8ac31f-2c2c-4126-8260-30ecdaea9298'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:41.048921 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-1d492dac-fc89-49d9-8777-f9977267ab3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a8961c2ccef4a5c9d73560037a23a60'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:41.049517 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-1d492dac-fc89-49d9-8777-f9977267ab3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a8961c2ccef4a5c9d73560037a23a60'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:41.064185 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ad8ac31f-2c2c-4126-8260-30ecdaea9298 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0a865a35946438d825a8bb09ed83944', 'request_id': 'req-ad8ac31f-2c2c-4126-8260-30ecdaea9298'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:41.073886 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ad8ac31f-2c2c-4126-8260-30ecdaea9298 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0a865a35946438d825a8bb09ed83944', 'request_id': 'req-ad8ac31f-2c2c-4126-8260-30ecdaea9298'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:41.092148 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 880/1771] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:43:40 2022] DELETE /identity/v3/users/4a8961c2ccef4a5c9d73560037a23a60 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:41.093537 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d492dac-fc89-49d9-8777-f9977267ab3a admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:41.096088 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-1d492dac-fc89-49d9-8777-f9977267ab3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:41.096505 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ad8ac31f-2c2c-4126-8260-30ecdaea9298 admin admin] The token cache is being invalidate because project b0a865a35946438d825a8bb09ed83944 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:43:41.096974 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ad8ac31f-2c2c-4126-8260-30ecdaea9298 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:41.098074 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ad8ac31f-2c2c-4126-8260-30ecdaea9298 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:41.103981 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 892/1772] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:43:40 2022] DELETE /identity/v3/projects/b0a865a35946438d825a8bb09ed83944 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:43:41.103981 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ac31f-2c2c-4126-8260-30ecdaea9298 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:41.107511 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-ad8ac31f-2c2c-4126-8260-30ecdaea9298 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:41.198162 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d492dac-fc89-49d9-8777-f9977267ab3a admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:41.200825 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ac31f-2c2c-4126-8260-30ecdaea9298 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:41.203313 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38020a1a-46c1-4d3e-9b7d-a38eccb2e4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:41.204199 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38020a1a-46c1-4d3e-9b7d-a38eccb2e4e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:41.204620 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38020a1a-46c1-4d3e-9b7d-a38eccb2e4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:41.204907 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38020a1a-46c1-4d3e-9b7d-a38eccb2e4e0 admin admin] PATH_INFO: `/v3/projects/30c09720daa44043b23e1c7e69012537` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:41.206163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38020a1a-46c1-4d3e-9b7d-a38eccb2e4e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30c09720daa44043b23e1c7e69012537)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:41.206985 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2988a3-098d-4ed9-b0c6-389ce589a724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:41.207945 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2988a3-098d-4ed9-b0c6-389ce589a724 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:41.208088 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38020a1a-46c1-4d3e-9b7d-a38eccb2e4e0 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:41.208436 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2988a3-098d-4ed9-b0c6-389ce589a724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:41.208869 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2988a3-098d-4ed9-b0c6-389ce589a724 admin admin] PATH_INFO: `/v3/projects/94c779535749491bbda14ec0f6ed0fd6` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:41.210606 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2988a3-098d-4ed9-b0c6-389ce589a724 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94c779535749491bbda14ec0f6ed0fd6)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:41.213433 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2988a3-098d-4ed9-b0c6-389ce589a724 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:41.265389 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-38020a1a-46c1-4d3e-9b7d-a38eccb2e4e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30c09720daa44043b23e1c7e69012537', 'request_id': 'req-38020a1a-46c1-4d3e-9b7d-a38eccb2e4e0'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:41.272064 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-38020a1a-46c1-4d3e-9b7d-a38eccb2e4e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30c09720daa44043b23e1c7e69012537', 'request_id': 'req-38020a1a-46c1-4d3e-9b7d-a38eccb2e4e0'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:41.272791 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6c2988a3-098d-4ed9-b0c6-389ce589a724 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94c779535749491bbda14ec0f6ed0fd6', 'request_id': 'req-6c2988a3-098d-4ed9-b0c6-389ce589a724'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:41.278524 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-38020a1a-46c1-4d3e-9b7d-a38eccb2e4e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30c09720daa44043b23e1c7e69012537', 'request_id': 'req-38020a1a-46c1-4d3e-9b7d-a38eccb2e4e0'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:41.283309 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6c2988a3-098d-4ed9-b0c6-389ce589a724 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94c779535749491bbda14ec0f6ed0fd6', 'request_id': 'req-6c2988a3-098d-4ed9-b0c6-389ce589a724'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:41.290700 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6c2988a3-098d-4ed9-b0c6-389ce589a724 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94c779535749491bbda14ec0f6ed0fd6', 'request_id': 'req-6c2988a3-098d-4ed9-b0c6-389ce589a724'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:41.296959 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-38020a1a-46c1-4d3e-9b7d-a38eccb2e4e0 admin admin] The token cache is being invalidate because project 30c09720daa44043b23e1c7e69012537 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:43:41.297320 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-38020a1a-46c1-4d3e-9b7d-a38eccb2e4e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:41.298143 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-38020a1a-46c1-4d3e-9b7d-a38eccb2e4e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:41.299833 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 881/1773] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:43:41 2022] DELETE /identity/v3/projects/30c09720daa44043b23e1c7e69012537 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:41.301057 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1e774-5d2e-4305-b1b9-2400e3159d33 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:41.301372 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1e774-5d2e-4305-b1b9-2400e3159d33 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:41.301654 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1e774-5d2e-4305-b1b9-2400e3159d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:41.308073 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6c2988a3-098d-4ed9-b0c6-389ce589a724 admin admin] The token cache is being invalidate because project 94c779535749491bbda14ec0f6ed0fd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:43:41.309091 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6c2988a3-098d-4ed9-b0c6-389ce589a724 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:41.309702 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6c2988a3-098d-4ed9-b0c6-389ce589a724 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:41.313246 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 893/1774] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:43:41 2022] DELETE /identity/v3/projects/94c779535749491bbda14ec0f6ed0fd6 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:41.316596 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a2801-42f1-4764-a026-16f10a8247e9 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:41.316892 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a2801-42f1-4764-a026-16f10a8247e9 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:41.317200 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a2801-42f1-4764-a026-16f10a8247e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:41.320264 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-94d1e774-5d2e-4305-b1b9-2400e3159d33 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:41.321606 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-94d1e774-5d2e-4305-b1b9-2400e3159d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:41.332812 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:41.337695 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-fb6a2801-42f1-4764-a026-16f10a8247e9 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:41.339060 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-fb6a2801-42f1-4764-a026-16f10a8247e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:41.399376 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 882/1775] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:41.399713 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d1e774-5d2e-4305-b1b9-2400e3159d33 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:41.401867 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-94d1e774-5d2e-4305-b1b9-2400e3159d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:41.418369 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 894/1776] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:41.419732 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e176205-554e-4c09-93f4-4e82448e19bd None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:41.420030 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e176205-554e-4c09-93f4-4e82448e19bd None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:41.420174 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d1e774-5d2e-4305-b1b9-2400e3159d33 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:41.420340 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e176205-554e-4c09-93f4-4e82448e19bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:41.426254 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd4f84d-a100-4c12-9ad2-2bb967931faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:41.427225 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4f84d-a100-4c12-9ad2-2bb967931faa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:41.427584 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4f84d-a100-4c12-9ad2-2bb967931faa admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:41.428064 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4f84d-a100-4c12-9ad2-2bb967931faa admin admin] PATH_INFO: `/v3/users/3438bf7f1d164791adf02ae7efe03c9c` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:41.428695 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:41.429399 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd4f84d-a100-4c12-9ad2-2bb967931faa admin admin] RBAC: Authorizing `identity:delete_user(user_id=3438bf7f1d164791adf02ae7efe03c9c)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:41.431379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd4f84d-a100-4c12-9ad2-2bb967931faa admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:41.456712 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-2e176205-554e-4c09-93f4-4e82448e19bd None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:41.458661 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-2e176205-554e-4c09-93f4-4e82448e19bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:41.478589 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 895/1777] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:41.479570 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e176205-554e-4c09-93f4-4e82448e19bd None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:41.488425 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-2e176205-554e-4c09-93f4-4e82448e19bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:41.521626 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-1dd4f84d-a100-4c12-9ad2-2bb967931faa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3438bf7f1d164791adf02ae7efe03c9c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:41.523147 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-1dd4f84d-a100-4c12-9ad2-2bb967931faa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3438bf7f1d164791adf02ae7efe03c9c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:41.550167 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-1dd4f84d-a100-4c12-9ad2-2bb967931faa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3438bf7f1d164791adf02ae7efe03c9c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:41.569447 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-1dd4f84d-a100-4c12-9ad2-2bb967931faa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3438bf7f1d164791adf02ae7efe03c9c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:41.570422 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-1dd4f84d-a100-4c12-9ad2-2bb967931faa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3438bf7f1d164791adf02ae7efe03c9c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:41.584251 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e176205-554e-4c09-93f4-4e82448e19bd None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:41.585520 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 883/1778] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:43:41 2022] DELETE /identity/v3/users/3438bf7f1d164791adf02ae7efe03c9c => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:41.587294 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e85417-be26-45ae-8d13-5569a00c0efb None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:41.587697 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e85417-be26-45ae-8d13-5569a00c0efb None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:41.588589 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e85417-be26-45ae-8d13-5569a00c0efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:41.614948 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-36e85417-be26-45ae-8d13-5569a00c0efb None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:41.616639 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-36e85417-be26-45ae-8d13-5569a00c0efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:41.681514 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c2866f-4c34-4628-b994-bb4ef10f3572 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:41.682475 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c2866f-4c34-4628-b994-bb4ef10f3572 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:41.682819 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c2866f-4c34-4628-b994-bb4ef10f3572 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:41.683198 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c2866f-4c34-4628-b994-bb4ef10f3572 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:41.686231 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-76c2866f-4c34-4628-b994-bb4ef10f3572 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:41.727151 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 884/1779] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:41.728821 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12831fc4-ef9a-468f-be85-6fe4cbf65492 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:41.729118 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12831fc4-ef9a-468f-be85-6fe4cbf65492 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:41.729583 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12831fc4-ef9a-468f-be85-6fe4cbf65492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:41.749978 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-12831fc4-ef9a-468f-be85-6fe4cbf65492 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:41.751761 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-12831fc4-ef9a-468f-be85-6fe4cbf65492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:41.760571 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 885/1780] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:41.761703 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12831fc4-ef9a-468f-be85-6fe4cbf65492 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:41.773836 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12831fc4-ef9a-468f-be85-6fe4cbf65492 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:41.784133 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e444815f-75f3-489d-9c2a-25f6308a0e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:41.785437 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444815f-75f3-489d-9c2a-25f6308a0e7c admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:41.786566 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444815f-75f3-489d-9c2a-25f6308a0e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:41.786986 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444815f-75f3-489d-9c2a-25f6308a0e7c admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:41.787889 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e444815f-75f3-489d-9c2a-25f6308a0e7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:43:41.788353 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e444815f-75f3-489d-9c2a-25f6308a0e7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:41.791889 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e444815f-75f3-489d-9c2a-25f6308a0e7c admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:41.793182 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c2866f-4c34-4628-b994-bb4ef10f3572 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:41.794873 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c2866f-4c34-4628-b994-bb4ef10f3572 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:41.809413 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 896/1781] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:43:41.810380 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c2866f-4c34-4628-b994-bb4ef10f3572 service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:41.813322 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 886/1782] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:43:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:43:41.817999 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e444815f-75f3-489d-9c2a-25f6308a0e7c admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:41.829356 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c2866f-4c34-4628-b994-bb4ef10f3572 service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:41.830124 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e444815f-75f3-489d-9c2a-25f6308a0e7c admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:41.834460 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08565e0-9b2a-472d-a995-ce13f1e1017c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:41.835224 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08565e0-9b2a-472d-a995-ce13f1e1017c admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:41.835331 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fa2b89-04bb-4c36-8627-7bae9885d73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:41.835588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08565e0-9b2a-472d-a995-ce13f1e1017c admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:41.836002 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08565e0-9b2a-472d-a995-ce13f1e1017c admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:41.836104 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa2b89-04bb-4c36-8627-7bae9885d73a admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:41.836443 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa2b89-04bb-4c36-8627-7bae9885d73a admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:41.836741 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa2b89-04bb-4c36-8627-7bae9885d73a admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:41.837477 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08565e0-9b2a-472d-a995-ce13f1e1017c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:43:41.837594 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fa2b89-04bb-4c36-8627-7bae9885d73a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:41.837880 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08565e0-9b2a-472d-a995-ce13f1e1017c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:41.839671 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fa2b89-04bb-4c36-8627-7bae9885d73a admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:41.840178 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08565e0-9b2a-472d-a995-ce13f1e1017c admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:41.846089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:41.866167 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 897/1783] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:43:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:43:41.867225 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08565e0-9b2a-472d-a995-ce13f1e1017c admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:41.881582 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08565e0-9b2a-472d-a995-ce13f1e1017c admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:41.889268 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b89dad-bb51-4829-a6e9-07e82fcdfa41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:41.890373 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b89dad-bb51-4829-a6e9-07e82fcdfa41 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:41.891834 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b89dad-bb51-4829-a6e9-07e82fcdfa41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:41.892304 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b89dad-bb51-4829-a6e9-07e82fcdfa41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:41.894263 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 887/1784] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:43:41 2022] POST /identity/v3/projects => generated 401 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:43:41.894263 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fa2b89-04bb-4c36-8627-7bae9885d73a admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:41.896076 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-d3fa2b89-04bb-4c36-8627-7bae9885d73a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:41.909398 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b89dad-bb51-4829-a6e9-07e82fcdfa41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:41.923160 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b89dad-bb51-4829-a6e9-07e82fcdfa41 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:41.979301 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 898/1785] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:43:41 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:43:41.979301 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b89dad-bb51-4829-a6e9-07e82fcdfa41 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:42.019886 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:42.031449 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fa2b89-04bb-4c36-8627-7bae9885d73a admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:42.075423 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b89dad-bb51-4829-a6e9-07e82fcdfa41 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:42.083823 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc68116-bd08-4ae2-845d-543f0fc22011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:42.084962 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc68116-bd08-4ae2-845d-543f0fc22011 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:42.085437 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc68116-bd08-4ae2-845d-543f0fc22011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:42.085871 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc68116-bd08-4ae2-845d-543f0fc22011 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:42.087504 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc68116-bd08-4ae2-845d-543f0fc22011 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:42.091302 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc68116-bd08-4ae2-845d-543f0fc22011 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:42.127267 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6854b65c-90c8-4e2c-be96-b85e4f9c0312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:42.128431 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854b65c-90c8-4e2c-be96-b85e4f9c0312 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:42.128868 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854b65c-90c8-4e2c-be96-b85e4f9c0312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:42.129386 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854b65c-90c8-4e2c-be96-b85e4f9c0312 admin admin] PATH_INFO: `/v3/projects/81a6110a2ddc4d69a76656dd25b565b2` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:42.131049 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6854b65c-90c8-4e2c-be96-b85e4f9c0312 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81a6110a2ddc4d69a76656dd25b565b2)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:42.134409 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6854b65c-90c8-4e2c-be96-b85e4f9c0312 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:42.137341 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 899/1786] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:43:41 2022] POST /identity/v3/users => generated 301 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:43:42.139477 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc68116-bd08-4ae2-845d-543f0fc22011 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:42.160500 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc68116-bd08-4ae2-845d-543f0fc22011 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:42.170160 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b67358-4e5b-4f35-88ea-fa6dbf302301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:42.171651 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b67358-4e5b-4f35-88ea-fa6dbf302301 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:42.174217 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b67358-4e5b-4f35-88ea-fa6dbf302301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:42.174518 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b67358-4e5b-4f35-88ea-fa6dbf302301 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:42.176310 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b67358-4e5b-4f35-88ea-fa6dbf302301 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:42.181401 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b67358-4e5b-4f35-88ea-fa6dbf302301 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:42.209185 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-6854b65c-90c8-4e2c-be96-b85e4f9c0312 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81a6110a2ddc4d69a76656dd25b565b2', 'request_id': 'req-6854b65c-90c8-4e2c-be96-b85e4f9c0312'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:42.220599 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 900/1787] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:43:42 2022] POST /identity/v3/users => generated 293 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:43:42.220599 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b67358-4e5b-4f35-88ea-fa6dbf302301 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:42.229390 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-6854b65c-90c8-4e2c-be96-b85e4f9c0312 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81a6110a2ddc4d69a76656dd25b565b2', 'request_id': 'req-6854b65c-90c8-4e2c-be96-b85e4f9c0312'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:42.233385 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b67358-4e5b-4f35-88ea-fa6dbf302301 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:42.241460 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-6854b65c-90c8-4e2c-be96-b85e4f9c0312 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81a6110a2ddc4d69a76656dd25b565b2', 'request_id': 'req-6854b65c-90c8-4e2c-be96-b85e4f9c0312'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:42.269740 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-6854b65c-90c8-4e2c-be96-b85e4f9c0312 admin admin] The token cache is being invalidate because project 81a6110a2ddc4d69a76656dd25b565b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:43:42.270043 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-6854b65c-90c8-4e2c-be96-b85e4f9c0312 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:42.272329 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-6854b65c-90c8-4e2c-be96-b85e4f9c0312 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:42.275767 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 888/1788] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:43:41 2022] DELETE /identity/v3/projects/81a6110a2ddc4d69a76656dd25b565b2 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:42.277427 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6854b65c-90c8-4e2c-be96-b85e4f9c0312 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:42.280297 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-6854b65c-90c8-4e2c-be96-b85e4f9c0312 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:42.319436 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:42.369776 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3797a7b8-9b16-4578-ae17-373ed7386585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:42.370879 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3797a7b8-9b16-4578-ae17-373ed7386585 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:42.371389 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3797a7b8-9b16-4578-ae17-373ed7386585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:42.371734 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3797a7b8-9b16-4578-ae17-373ed7386585 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:42.372580 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3797a7b8-9b16-4578-ae17-373ed7386585 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:42.375008 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3797a7b8-9b16-4578-ae17-373ed7386585 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:42.385079 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6854b65c-90c8-4e2c-be96-b85e4f9c0312 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:42.387096 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 901/1789] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:42 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:43:42.391971 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd67869-5c3d-458a-ae4a-945d8457f8ca None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:42.392349 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd67869-5c3d-458a-ae4a-945d8457f8ca None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:42.392754 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd67869-5c3d-458a-ae4a-945d8457f8ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:42.393687 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171295a-67c4-4837-8155-2f3ef8437ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:42.394451 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171295a-67c4-4837-8155-2f3ef8437ab5 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:42.394813 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171295a-67c4-4837-8155-2f3ef8437ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:42.395165 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171295a-67c4-4837-8155-2f3ef8437ab5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:42.396397 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9171295a-67c4-4837-8155-2f3ef8437ab5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:42.399350 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9171295a-67c4-4837-8155-2f3ef8437ab5 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:42.416809 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 889/1790] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:42 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:43:42.417952 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171295a-67c4-4837-8155-2f3ef8437ab5 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:42.420839 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-9171295a-67c4-4837-8155-2f3ef8437ab5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:42.426934 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-cdd67869-5c3d-458a-ae4a-945d8457f8ca None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:42.431506 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-cdd67869-5c3d-458a-ae4a-945d8457f8ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:42.446935 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171295a-67c4-4837-8155-2f3ef8437ab5 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:42.449312 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 902/1791] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:42.449921 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd67869-5c3d-458a-ae4a-945d8457f8ca None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:42.461729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd2b73d-e7ee-476a-834b-d42016a326e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:42.462422 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd2b73d-e7ee-476a-834b-d42016a326e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:42.462714 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd2b73d-e7ee-476a-834b-d42016a326e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:42.462996 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd2b73d-e7ee-476a-834b-d42016a326e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:42.463770 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd2b73d-e7ee-476a-834b-d42016a326e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:42.465462 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd2b73d-e7ee-476a-834b-d42016a326e6 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:42.474426 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd67869-5c3d-458a-ae4a-945d8457f8ca None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:42.481899 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 890/1792] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:42 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:43:42.484425 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c06a7-e78e-482a-8bfe-bea19ac98b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:42.485968 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c06a7-e78e-482a-8bfe-bea19ac98b0a admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:42.486526 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c06a7-e78e-482a-8bfe-bea19ac98b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:42.487023 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c06a7-e78e-482a-8bfe-bea19ac98b0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:42.487335 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898deeb1-8e03-45b8-9343-a683a8f19cfd None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:42.487765 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898deeb1-8e03-45b8-9343-a683a8f19cfd None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:42.487903 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7c06a7-e78e-482a-8bfe-bea19ac98b0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:42.488093 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898deeb1-8e03-45b8-9343-a683a8f19cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:42.494394 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7c06a7-e78e-482a-8bfe-bea19ac98b0a admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:42.519867 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-898deeb1-8e03-45b8-9343-a683a8f19cfd None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:42.520309 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 903/1793] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:42 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:43:42.521740 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-898deeb1-8e03-45b8-9343-a683a8f19cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:42.524865 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c06a7-e78e-482a-8bfe-bea19ac98b0a admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:42.536239 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c06a7-e78e-482a-8bfe-bea19ac98b0a admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:42.536696 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 891/1794] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:42.537845 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898deeb1-8e03-45b8-9343-a683a8f19cfd None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:42.539423 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-898deeb1-8e03-45b8-9343-a683a8f19cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:42.541594 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82821a9-b399-436a-8abd-5cff2a7c3dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:42.542565 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82821a9-b399-436a-8abd-5cff2a7c3dfe admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:42.542926 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82821a9-b399-436a-8abd-5cff2a7c3dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:42.543332 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82821a9-b399-436a-8abd-5cff2a7c3dfe admin admin] PATH_INFO: `/v3/projects/217bf21d2bda40d99a25b20793f0e835/users/913b81ca38a6430a95597d70ebe42a0a/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:42.555182 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898deeb1-8e03-45b8-9343-a683a8f19cfd None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:42.560192 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3532e93-4eb4-4963-a61a-9468e806a14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:42.561235 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3532e93-4eb4-4963-a61a-9468e806a14d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:42.561532 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3532e93-4eb4-4963-a61a-9468e806a14d admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:42.561788 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3532e93-4eb4-4963-a61a-9468e806a14d admin admin] PATH_INFO: `/v3/projects/3197c37a5080426bae587174668b496d` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:42.563040 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3532e93-4eb4-4963-a61a-9468e806a14d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3197c37a5080426bae587174668b496d)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:42.564306 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82821a9-b399-436a-8abd-5cff2a7c3dfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=217bf21d2bda40d99a25b20793f0e835, user_id=913b81ca38a6430a95597d70ebe42a0a, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:42.564962 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3532e93-4eb4-4963-a61a-9468e806a14d admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:42.567869 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82821a9-b399-436a-8abd-5cff2a7c3dfe admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:42.592539 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 904/1795] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:43:42 2022] PUT /identity/v3/projects/217bf21d2bda40d99a25b20793f0e835/users/913b81ca38a6430a95597d70ebe42a0a/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:42.592539 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82821a9-b399-436a-8abd-5cff2a7c3dfe admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:42.633339 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-a3532e93-4eb4-4963-a61a-9468e806a14d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3197c37a5080426bae587174668b496d', 'request_id': 'req-a3532e93-4eb4-4963-a61a-9468e806a14d'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:42.640300 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-a3532e93-4eb4-4963-a61a-9468e806a14d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3197c37a5080426bae587174668b496d', 'request_id': 'req-a3532e93-4eb4-4963-a61a-9468e806a14d'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:42.647155 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-a3532e93-4eb4-4963-a61a-9468e806a14d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3197c37a5080426bae587174668b496d', 'request_id': 'req-a3532e93-4eb4-4963-a61a-9468e806a14d'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:42.650444 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:42.662914 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-a3532e93-4eb4-4963-a61a-9468e806a14d admin admin] The token cache is being invalidate because project 3197c37a5080426bae587174668b496d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:43:42.663379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-a3532e93-4eb4-4963-a61a-9468e806a14d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:42.664275 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-a3532e93-4eb4-4963-a61a-9468e806a14d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:42.668249 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 892/1796] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:43:42 2022] DELETE /identity/v3/projects/3197c37a5080426bae587174668b496d => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:42.670038 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82821a9-b399-436a-8abd-5cff2a7c3dfe admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:42.670341 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3532e93-4eb4-4963-a61a-9468e806a14d admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:42.672644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-a3532e93-4eb4-4963-a61a-9468e806a14d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:42.768222 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3532e93-4eb4-4963-a61a-9468e806a14d admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:42.774356 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c9a89-5da4-4205-8011-f37681e2273d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:42.775052 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c9a89-5da4-4205-8011-f37681e2273d admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:42.775401 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c9a89-5da4-4205-8011-f37681e2273d admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:42.775694 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c9a89-5da4-4205-8011-f37681e2273d admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:42.776697 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3c9a89-5da4-4205-8011-f37681e2273d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:43:42.777128 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3c9a89-5da4-4205-8011-f37681e2273d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:42.779595 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3c9a89-5da4-4205-8011-f37681e2273d admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:42.796829 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 893/1797] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:43:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:43:42.796829 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce26bf-6024-438a-bd3f-8ea0b12273e4 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:42.796829 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce26bf-6024-438a-bd3f-8ea0b12273e4 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:42.796829 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce26bf-6024-438a-bd3f-8ea0b12273e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:42.815048 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9ce72-c200-4833-b14d-6873894103a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:42.816493 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9ce72-c200-4833-b14d-6873894103a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:42.816493 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9ce72-c200-4833-b14d-6873894103a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:42.816848 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9ce72-c200-4833-b14d-6873894103a8 admin admin] PATH_INFO: `/v3/projects/a93efcc3f25a496a9a093d6835ce3cb3/users/45ac490771d745a98754be293cceb749/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:42.831588 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-45ce26bf-6024-438a-bd3f-8ea0b12273e4 None None] MFA Rules not processed for user `913b81ca38a6430a95597d70ebe42a0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:42.832993 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-45ce26bf-6024-438a-bd3f-8ea0b12273e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:42.838840 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde9ce72-c200-4833-b14d-6873894103a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a93efcc3f25a496a9a093d6835ce3cb3, user_id=45ac490771d745a98754be293cceb749, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:42.843111 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde9ce72-c200-4833-b14d-6873894103a8 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:42.843240 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:42.866866 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 905/1798] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:43:42 2022] PUT /identity/v3/projects/a93efcc3f25a496a9a093d6835ce3cb3/users/45ac490771d745a98754be293cceb749/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:42.868271 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26fa8ff-b2eb-4e7d-a693-a53ad815a148 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:42.868568 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26fa8ff-b2eb-4e7d-a693-a53ad815a148 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:42.868855 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26fa8ff-b2eb-4e7d-a693-a53ad815a148 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:42.870211 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 906/1799] 10.222.0.36 () {54 vars in 921 bytes} [Thu May 19 16:43:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:43:42.871303 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62659506-7e79-42d6-9fe3-3795aaf86e84 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:42.871591 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62659506-7e79-42d6-9fe3-3795aaf86e84 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:42.871880 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62659506-7e79-42d6-9fe3-3795aaf86e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:42.895221 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-62659506-7e79-42d6-9fe3-3795aaf86e84 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:42.896479 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-62659506-7e79-42d6-9fe3-3795aaf86e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:42.951117 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 894/1800] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:42 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:42.952329 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ce26bf-6024-438a-bd3f-8ea0b12273e4 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:42.964969 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ce26bf-6024-438a-bd3f-8ea0b12273e4 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:43.004620 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 907/1801] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:43.006837 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a7567-ead7-4006-89a8-8a201af87c5b None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:43.007237 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a7567-ead7-4006-89a8-8a201af87c5b None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:43.007624 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a7567-ead7-4006-89a8-8a201af87c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:43.045649 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-652a7567-ead7-4006-89a8-8a201af87c5b None None] MFA Rules not processed for user `45ac490771d745a98754be293cceb749`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:43.047097 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab6d1f-9353-45b8-a79b-d6fabe1efff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:43.047263 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-652a7567-ead7-4006-89a8-8a201af87c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:43.047931 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab6d1f-9353-45b8-a79b-d6fabe1efff4 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:43.048898 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab6d1f-9353-45b8-a79b-d6fabe1efff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:43.048898 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab6d1f-9353-45b8-a79b-d6fabe1efff4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:43.050007 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ab6d1f-9353-45b8-a79b-d6fabe1efff4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:43.052447 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ab6d1f-9353-45b8-a79b-d6fabe1efff4 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:43.088719 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 895/1802] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:43:42 2022] POST /identity/v3/projects => generated 393 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:43:43.090583 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8e5cd-848d-4674-ad26-6def6cad9966 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:43.090955 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8e5cd-848d-4674-ad26-6def6cad9966 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:43.091487 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8e5cd-848d-4674-ad26-6def6cad9966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:43.114764 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-51c8e5cd-848d-4674-ad26-6def6cad9966 None None] MFA Rules not processed for user `58bdeada00b74b7b8e186e6dc51e6dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:43.116344 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-51c8e5cd-848d-4674-ad26-6def6cad9966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:43.142269 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 908/1803] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:43.142269 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a7567-ead7-4006-89a8-8a201af87c5b None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:43.142269 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-652a7567-ead7-4006-89a8-8a201af87c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:43.170775 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a7567-ead7-4006-89a8-8a201af87c5b None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:43.186165 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:43.257713 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec1724-b8d3-4c86-96c4-217cf8490529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:43.259192 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec1724-b8d3-4c86-96c4-217cf8490529 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:43.259834 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec1724-b8d3-4c86-96c4-217cf8490529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:43.260278 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec1724-b8d3-4c86-96c4-217cf8490529 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:43.261553 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ec1724-b8d3-4c86-96c4-217cf8490529 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:43.264099 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ec1724-b8d3-4c86-96c4-217cf8490529 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:43.268509 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 896/1804] 10.222.0.36 () {58 vars in 1018 bytes} [Thu May 19 16:43:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:43.270705 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800a8fc-4e09-4d96-9acc-2f49927ed939 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:43.272202 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800a8fc-4e09-4d96-9acc-2f49927ed939 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:43.272573 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800a8fc-4e09-4d96-9acc-2f49927ed939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:43.310025 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 909/1805] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:43:43 2022] POST /identity/v3/projects => generated 401 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:43:43.310368 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-a800a8fc-4e09-4d96-9acc-2f49927ed939 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:43.310687 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec1724-b8d3-4c86-96c4-217cf8490529 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:43.311817 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-a800a8fc-4e09-4d96-9acc-2f49927ed939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:43.342365 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:43.391974 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 897/1806] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:43.393052 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800a8fc-4e09-4d96-9acc-2f49927ed939 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:43.393480 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec1724-b8d3-4c86-96c4-217cf8490529 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:43.396214 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-a800a8fc-4e09-4d96-9acc-2f49927ed939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:43.399531 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a505436-9bb5-41b5-a82c-0cec0c9ae204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:43.406819 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a505436-9bb5-41b5-a82c-0cec0c9ae204 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:43.407194 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a505436-9bb5-41b5-a82c-0cec0c9ae204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:43.407605 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a505436-9bb5-41b5-a82c-0cec0c9ae204 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:43.408411 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a505436-9bb5-41b5-a82c-0cec0c9ae204 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:43.412340 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a505436-9bb5-41b5-a82c-0cec0c9ae204 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:43.415907 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800a8fc-4e09-4d96-9acc-2f49927ed939 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:43.424961 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d975d7-9159-4fcd-a10a-0e608f35bf72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:43.426033 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d975d7-9159-4fcd-a10a-0e608f35bf72 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:43.426359 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d975d7-9159-4fcd-a10a-0e608f35bf72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:43.426924 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d975d7-9159-4fcd-a10a-0e608f35bf72 admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:43.427801 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d975d7-9159-4fcd-a10a-0e608f35bf72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:43.430621 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d975d7-9159-4fcd-a10a-0e608f35bf72 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:43.443677 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 910/1807] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:43:43 2022] POST /identity/v3/users => generated 297 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:43:43.443677 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a505436-9bb5-41b5-a82c-0cec0c9ae204 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:43.445987 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-9a505436-9bb5-41b5-a82c-0cec0c9ae204 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:43.468781 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 898/1808] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:43:43 2022] POST /identity/v3/projects => generated 385 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:43:43.468781 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d975d7-9159-4fcd-a10a-0e608f35bf72 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:43.578828 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d975d7-9159-4fcd-a10a-0e608f35bf72 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:43.581196 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a505436-9bb5-41b5-a82c-0cec0c9ae204 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:43.584246 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b16167-b26c-4e01-be89-8005f0d40546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:43.584791 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b16167-b26c-4e01-be89-8005f0d40546 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:43.585190 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b16167-b26c-4e01-be89-8005f0d40546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:43.585653 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b16167-b26c-4e01-be89-8005f0d40546 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:43.586741 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b16167-b26c-4e01-be89-8005f0d40546 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:43.588853 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b16167-b26c-4e01-be89-8005f0d40546 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:43.613478 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:43.621843 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 899/1809] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:43:43 2022] POST /identity/v3/users => generated 307 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:43:43.623078 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b16167-b26c-4e01-be89-8005f0d40546 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:43.639219 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b16167-b26c-4e01-be89-8005f0d40546 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:43.646359 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdf050c-b4a5-411b-9f78-b0f905bf7023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:43.647435 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf050c-b4a5-411b-9f78-b0f905bf7023 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:43.647731 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf050c-b4a5-411b-9f78-b0f905bf7023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:43.648013 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf050c-b4a5-411b-9f78-b0f905bf7023 admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:43.648815 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdf050c-b4a5-411b-9f78-b0f905bf7023 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:43:43.649309 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdf050c-b4a5-411b-9f78-b0f905bf7023 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:43.651903 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdf050c-b4a5-411b-9f78-b0f905bf7023 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:43.670787 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe6c7ba-650d-4bd6-8309-e24c44d1b256 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:43.671645 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6c7ba-650d-4bd6-8309-e24c44d1b256 service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:43.672578 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6c7ba-650d-4bd6-8309-e24c44d1b256 service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:43.673104 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6c7ba-650d-4bd6-8309-e24c44d1b256 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:43.680267 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 900/1810] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:43:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:43:43.681133 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdf050c-b4a5-411b-9f78-b0f905bf7023 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:43.689440 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe6c7ba-650d-4bd6-8309-e24c44d1b256 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:43.691924 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe6c7ba-650d-4bd6-8309-e24c44d1b256 service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:43.702142 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdf050c-b4a5-411b-9f78-b0f905bf7023 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:43.708197 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b350bdc-f4bc-4229-8df0-f873425faff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:43.708991 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b350bdc-f4bc-4229-8df0-f873425faff3 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:43.709754 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b350bdc-f4bc-4229-8df0-f873425faff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:43.709943 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 911/1811] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:43:43.710026 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b350bdc-f4bc-4229-8df0-f873425faff3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:43.710933 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b350bdc-f4bc-4229-8df0-f873425faff3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:43.712279 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b350bdc-f4bc-4229-8df0-f873425faff3 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:43.713057 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe6c7ba-650d-4bd6-8309-e24c44d1b256 service nova] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:43.729759 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 901/1812] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:43 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:43:43.730885 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b350bdc-f4bc-4229-8df0-f873425faff3 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:43.731390 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe6c7ba-650d-4bd6-8309-e24c44d1b256 service nova] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:43.740312 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9091c227-3ab9-4c45-ab20-b6902e11e0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:43.741436 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091c227-3ab9-4c45-ab20-b6902e11e0e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:43.741949 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091c227-3ab9-4c45-ab20-b6902e11e0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:43.742446 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091c227-3ab9-4c45-ab20-b6902e11e0e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:43.742551 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b350bdc-f4bc-4229-8df0-f873425faff3 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:43.743933 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9091c227-3ab9-4c45-ab20-b6902e11e0e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:43.746983 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9091c227-3ab9-4c45-ab20-b6902e11e0e4 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:43.747242 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e10e89-c9d0-4bae-acad-0fdf7d040a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:43.747921 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e10e89-c9d0-4bae-acad-0fdf7d040a36 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:43.748209 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e10e89-c9d0-4bae-acad-0fdf7d040a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:43.748488 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e10e89-c9d0-4bae-acad-0fdf7d040a36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:43.756922 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e10e89-c9d0-4bae-acad-0fdf7d040a36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:43.758611 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e10e89-c9d0-4bae-acad-0fdf7d040a36 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:43.786660 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 902/1813] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:43 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:43:43.787630 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e10e89-c9d0-4bae-acad-0fdf7d040a36 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:43.789631 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 912/1814] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:43:43 2022] POST /identity/v3/users => generated 299 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:43:43.790730 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9091c227-3ab9-4c45-ab20-b6902e11e0e4 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:43.799219 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e10e89-c9d0-4bae-acad-0fdf7d040a36 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:43.806406 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c070bc3b-8395-498c-85a9-332fc6c661e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:43.807130 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9091c227-3ab9-4c45-ab20-b6902e11e0e4 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:43.807289 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070bc3b-8395-498c-85a9-332fc6c661e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:43.807525 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070bc3b-8395-498c-85a9-332fc6c661e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:43.808240 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070bc3b-8395-498c-85a9-332fc6c661e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:43.808508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c070bc3b-8395-498c-85a9-332fc6c661e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:43.812079 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c070bc3b-8395-498c-85a9-332fc6c661e9 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:43.813815 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa199e-d346-4dae-b076-08d3299c2471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:43.814650 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa199e-d346-4dae-b076-08d3299c2471 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:43.815046 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa199e-d346-4dae-b076-08d3299c2471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:43.815601 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa199e-d346-4dae-b076-08d3299c2471 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:43.816746 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fa199e-d346-4dae-b076-08d3299c2471 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:43.818863 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fa199e-d346-4dae-b076-08d3299c2471 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:43.839492 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 913/1815] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:43 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:43:43.839492 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa199e-d346-4dae-b076-08d3299c2471 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:43.850071 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa199e-d346-4dae-b076-08d3299c2471 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:43.853043 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 903/1816] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:43:43 2022] POST /identity/v3/projects => generated 405 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:43:43.853989 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c070bc3b-8395-498c-85a9-332fc6c661e9 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:43.941642 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c99526-2dda-4987-8d1d-96576d092c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:43.943209 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c99526-2dda-4987-8d1d-96576d092c31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:43.943622 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c99526-2dda-4987-8d1d-96576d092c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:43.944002 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c99526-2dda-4987-8d1d-96576d092c31 admin admin] PATH_INFO: `/v3/projects/31b13cf1a01f425abb877ba397378765/users/4c77abae271e49908a89fc66747843d9/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:43.944459 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:43.972316 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c070bc3b-8395-498c-85a9-332fc6c661e9 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:43.975049 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c99526-2dda-4987-8d1d-96576d092c31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31b13cf1a01f425abb877ba397378765, user_id=4c77abae271e49908a89fc66747843d9, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:43.977127 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1bf44f-52e8-45b3-b51c-c4fc07c0858d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:43.977860 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1bf44f-52e8-45b3-b51c-c4fc07c0858d admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:43.978151 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1bf44f-52e8-45b3-b51c-c4fc07c0858d admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:43.978241 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c99526-2dda-4987-8d1d-96576d092c31 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:43.978439 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1bf44f-52e8-45b3-b51c-c4fc07c0858d admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:43.979190 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1bf44f-52e8-45b3-b51c-c4fc07c0858d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:43.980888 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1bf44f-52e8-45b3-b51c-c4fc07c0858d admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:43.996054 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 904/1817] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:43 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:43:43.996054 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1bf44f-52e8-45b3-b51c-c4fc07c0858d admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:43.996346 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 914/1818] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:43:43 2022] PUT /identity/v3/projects/31b13cf1a01f425abb877ba397378765/users/4c77abae271e49908a89fc66747843d9/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:43.996346 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c99526-2dda-4987-8d1d-96576d092c31 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:44.008583 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1bf44f-52e8-45b3-b51c-c4fc07c0858d admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:44.077923 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c99526-2dda-4987-8d1d-96576d092c31 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:44.084198 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6685d252-69b8-40f6-baad-fadb42a3e434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:44.084988 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98afaa47-a820-467c-a262-a568efe8b3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:44.085261 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6685d252-69b8-40f6-baad-fadb42a3e434 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:44.085614 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6685d252-69b8-40f6-baad-fadb42a3e434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:44.085723 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98afaa47-a820-467c-a262-a568efe8b3c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:44.085954 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98afaa47-a820-467c-a262-a568efe8b3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:44.086044 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6685d252-69b8-40f6-baad-fadb42a3e434 admin admin] PATH_INFO: `/v3/projects/7b4343c7f07f40e0a20bdd3c6720a757/users/eeddf9c1eea74b33b7e23f322ceee9e6/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:44.086233 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98afaa47-a820-467c-a262-a568efe8b3c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:44.086934 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98afaa47-a820-467c-a262-a568efe8b3c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:44.088875 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98afaa47-a820-467c-a262-a568efe8b3c8 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:44.099927 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:44.107826 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6685d252-69b8-40f6-baad-fadb42a3e434 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b4343c7f07f40e0a20bdd3c6720a757, user_id=eeddf9c1eea74b33b7e23f322ceee9e6, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:44.112455 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6685d252-69b8-40f6-baad-fadb42a3e434 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:44.116494 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 915/1819] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:43:43 2022] POST /identity/v3/users => generated 303 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:43:44.118063 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98afaa47-a820-467c-a262-a568efe8b3c8 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:44.130355 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98afaa47-a820-467c-a262-a568efe8b3c8 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:44.133001 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 905/1820] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:43:43 2022] PUT /identity/v3/projects/7b4343c7f07f40e0a20bdd3c6720a757/users/eeddf9c1eea74b33b7e23f322ceee9e6/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:44.141743 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6685d252-69b8-40f6-baad-fadb42a3e434 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:44.214887 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070faea2-a262-44df-b20b-a71163a1e225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:44.215987 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070faea2-a262-44df-b20b-a71163a1e225 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:44.216432 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070faea2-a262-44df-b20b-a71163a1e225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:44.216978 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070faea2-a262-44df-b20b-a71163a1e225 admin admin] PATH_INFO: `/v3/projects/8d15182c742047e19af57f2f4d466192/users/0ed942914a324076aaad268808717508/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:44.238868 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070faea2-a262-44df-b20b-a71163a1e225 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d15182c742047e19af57f2f4d466192, user_id=0ed942914a324076aaad268808717508, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:44.243435 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070faea2-a262-44df-b20b-a71163a1e225 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:44.260449 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 916/1821] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:43:44 2022] PUT /identity/v3/projects/8d15182c742047e19af57f2f4d466192/users/0ed942914a324076aaad268808717508/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:44.261581 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070faea2-a262-44df-b20b-a71163a1e225 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:44.271449 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6685d252-69b8-40f6-baad-fadb42a3e434 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:44.350561 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070faea2-a262-44df-b20b-a71163a1e225 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:44.352071 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ae2a60-eaea-476c-bc29-c09cab528d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:44.353054 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae2a60-eaea-476c-bc29-c09cab528d66 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:44.353971 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae2a60-eaea-476c-bc29-c09cab528d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:44.354273 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae2a60-eaea-476c-bc29-c09cab528d66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:44.355259 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ae2a60-eaea-476c-bc29-c09cab528d66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:44.356581 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd0730-8f82-4345-9626-5d8c4ae41b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:44.357303 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd0730-8f82-4345-9626-5d8c4ae41b83 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:44.357595 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd0730-8f82-4345-9626-5d8c4ae41b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:44.357878 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd0730-8f82-4345-9626-5d8c4ae41b83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:44.357981 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ae2a60-eaea-476c-bc29-c09cab528d66 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:44.358720 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bd0730-8f82-4345-9626-5d8c4ae41b83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:44.360493 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bd0730-8f82-4345-9626-5d8c4ae41b83 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:44.367307 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:44.379867 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 917/1822] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:43:44.379867 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ad408-401d-4f69-bc2e-f8f43fc786b7 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:44.379867 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ad408-401d-4f69-bc2e-f8f43fc786b7 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:44.379867 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ad408-401d-4f69-bc2e-f8f43fc786b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:44.383446 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 906/1823] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:43:44.384481 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ae2a60-eaea-476c-bc29-c09cab528d66 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:44.400006 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ae2a60-eaea-476c-bc29-c09cab528d66 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:44.406244 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a8774c-fc85-4bcf-a50d-31657e152ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:44.407363 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8774c-fc85-4bcf-a50d-31657e152ead admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:44.407699 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8774c-fc85-4bcf-a50d-31657e152ead admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:44.408053 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8774c-fc85-4bcf-a50d-31657e152ead admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:44.408895 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a8774c-fc85-4bcf-a50d-31657e152ead admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:44.410637 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a8774c-fc85-4bcf-a50d-31657e152ead admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:44.415014 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-354ad408-401d-4f69-bc2e-f8f43fc786b7 None None] MFA Rules not processed for user `eeddf9c1eea74b33b7e23f322ceee9e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:44.416337 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-354ad408-401d-4f69-bc2e-f8f43fc786b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:44.424763 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 907/1824] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:43:44.424763 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a8774c-fc85-4bcf-a50d-31657e152ead admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:44.436269 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a8774c-fc85-4bcf-a50d-31657e152ead admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:44.442937 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59cda4f-1de8-4e2a-a3e0-f1c9ddfd0ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:44.443781 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59cda4f-1de8-4e2a-a3e0-f1c9ddfd0ef9 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:44.444077 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59cda4f-1de8-4e2a-a3e0-f1c9ddfd0ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:44.444365 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59cda4f-1de8-4e2a-a3e0-f1c9ddfd0ef9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:44.445738 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59cda4f-1de8-4e2a-a3e0-f1c9ddfd0ef9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:44.447607 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59cda4f-1de8-4e2a-a3e0-f1c9ddfd0ef9 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:44.464765 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 908/1825] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:43:44.465760 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59cda4f-1de8-4e2a-a3e0-f1c9ddfd0ef9 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:44.480434 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59cda4f-1de8-4e2a-a3e0-f1c9ddfd0ef9 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:44.490434 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8031172c-70fe-4910-bf9a-6ed724a39fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:44.491548 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031172c-70fe-4910-bf9a-6ed724a39fa3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:44.492240 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031172c-70fe-4910-bf9a-6ed724a39fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:44.492635 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031172c-70fe-4910-bf9a-6ed724a39fa3 admin admin] PATH_INFO: `/v3/projects/31b13cf1a01f425abb877ba397378765/users/4c77abae271e49908a89fc66747843d9/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:44.494910 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8031172c-70fe-4910-bf9a-6ed724a39fa3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31b13cf1a01f425abb877ba397378765, user_id=4c77abae271e49908a89fc66747843d9, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:44.498522 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8031172c-70fe-4910-bf9a-6ed724a39fa3 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:44.502630 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 918/1826] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:44.504821 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ad408-401d-4f69-bc2e-f8f43fc786b7 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:44.524692 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 909/1827] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:43:44 2022] PUT /identity/v3/projects/31b13cf1a01f425abb877ba397378765/users/4c77abae271e49908a89fc66747843d9/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:44.524692 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8031172c-70fe-4910-bf9a-6ed724a39fa3 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:44.525214 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ad408-401d-4f69-bc2e-f8f43fc786b7 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:44.525760 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:44.613329 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8031172c-70fe-4910-bf9a-6ed724a39fa3 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:44.619796 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d07221-df64-44d0-97bb-690f61f76271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:44.620801 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d07221-df64-44d0-97bb-690f61f76271 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:44.621187 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d07221-df64-44d0-97bb-690f61f76271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:44.621537 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d07221-df64-44d0-97bb-690f61f76271 admin admin] PATH_INFO: `/v3/projects/c2c5934d7e484e42972680eb65656f14/users/9ca86917a040495ea48905d363c9e8d7/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:44.628236 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf559473-a245-4251-86d7-77b7861e5aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:44.630015 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf559473-a245-4251-86d7-77b7861e5aee admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:44.630463 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf559473-a245-4251-86d7-77b7861e5aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:44.630882 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf559473-a245-4251-86d7-77b7861e5aee admin admin] PATH_INFO: `/v3/projects/8d15182c742047e19af57f2f4d466192/users/0ed942914a324076aaad268808717508/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:44.635127 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf559473-a245-4251-86d7-77b7861e5aee admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d15182c742047e19af57f2f4d466192, user_id=0ed942914a324076aaad268808717508, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:44.640500 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf559473-a245-4251-86d7-77b7861e5aee admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:44.655133 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d07221-df64-44d0-97bb-690f61f76271 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2c5934d7e484e42972680eb65656f14, user_id=9ca86917a040495ea48905d363c9e8d7, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:44.660789 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d07221-df64-44d0-97bb-690f61f76271 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:44.662553 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 919/1828] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:43:44 2022] PUT /identity/v3/projects/8d15182c742047e19af57f2f4d466192/users/0ed942914a324076aaad268808717508/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:44.663166 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf559473-a245-4251-86d7-77b7861e5aee admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:44.691519 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 910/1829] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:43:44 2022] PUT /identity/v3/projects/c2c5934d7e484e42972680eb65656f14/users/9ca86917a040495ea48905d363c9e8d7/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:44.695220 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a1da0-bcc1-46b8-b34f-8b881f20ac42 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:44.695666 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a1da0-bcc1-46b8-b34f-8b881f20ac42 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:44.695965 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a1da0-bcc1-46b8-b34f-8b881f20ac42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:44.738469 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-110a1da0-bcc1-46b8-b34f-8b881f20ac42 None None] MFA Rules not processed for user `4c77abae271e49908a89fc66747843d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:44.739784 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-110a1da0-bcc1-46b8-b34f-8b881f20ac42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:44.762880 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf559473-a245-4251-86d7-77b7861e5aee admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:44.833577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 911/1830] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:44 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:44.833577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20276db4-6ec7-461d-847a-91e43d494ff7 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:44.833577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20276db4-6ec7-461d-847a-91e43d494ff7 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:44.833577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20276db4-6ec7-461d-847a-91e43d494ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:44.838386 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545537ce-1b18-4618-bc04-3de4904314e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:44.839627 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545537ce-1b18-4618-bc04-3de4904314e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:44.840620 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545537ce-1b18-4618-bc04-3de4904314e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:44.841045 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545537ce-1b18-4618-bc04-3de4904314e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:44.842063 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545537ce-1b18-4618-bc04-3de4904314e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:44.844265 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545537ce-1b18-4618-bc04-3de4904314e9 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:44.870599 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-20276db4-6ec7-461d-847a-91e43d494ff7 None None] MFA Rules not processed for user `0ed942914a324076aaad268808717508`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:44.872270 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-20276db4-6ec7-461d-847a-91e43d494ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:44.881691 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 920/1831] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:43:44 2022] POST /identity/v3/projects => generated 393 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:43:44.883643 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b574fa6-2a92-4586-b101-ff24590dbbd8 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:44.884047 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b574fa6-2a92-4586-b101-ff24590dbbd8 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:44.884704 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b574fa6-2a92-4586-b101-ff24590dbbd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:44.931944 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:44.952055 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-1b574fa6-2a92-4586-b101-ff24590dbbd8 None None] MFA Rules not processed for user `9ca86917a040495ea48905d363c9e8d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:44.956104 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-1b574fa6-2a92-4586-b101-ff24590dbbd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:44.988386 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:45.014872 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 912/1832] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:44 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:45.016793 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20276db4-6ec7-461d-847a-91e43d494ff7 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:45.018960 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-20276db4-6ec7-461d-847a-91e43d494ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:45.088242 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 921/1833] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:44 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:45.089857 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b574fa6-2a92-4586-b101-ff24590dbbd8 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:45.164396 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20276db4-6ec7-461d-847a-91e43d494ff7 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:45.176346 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0120b6-a344-4168-9761-34613fa3353b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:45.177023 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0120b6-a344-4168-9761-34613fa3353b service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:45.177341 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0120b6-a344-4168-9761-34613fa3353b service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:45.177625 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0120b6-a344-4168-9761-34613fa3353b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:45.210083 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b574fa6-2a92-4586-b101-ff24590dbbd8 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:45.219018 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdfa334-243c-4642-b5f6-641813817d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:45.219726 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdfa334-243c-4642-b5f6-641813817d48 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:45.220010 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdfa334-243c-4642-b5f6-641813817d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:45.220290 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdfa334-243c-4642-b5f6-641813817d48 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:45.221012 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdfa334-243c-4642-b5f6-641813817d48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:45.223030 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdfa334-243c-4642-b5f6-641813817d48 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:45.268792 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 922/1834] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:43:45 2022] POST /identity/v3/users => generated 303 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:43:45.268792 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdfa334-243c-4642-b5f6-641813817d48 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:45.280701 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-4fdfa334-243c-4642-b5f6-641813817d48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:45.294795 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdfa334-243c-4642-b5f6-641813817d48 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:45.304755 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644def7f-02d9-40d2-989f-5fde914f0c4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:45.305454 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644def7f-02d9-40d2-989f-5fde914f0c4f service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:45.305744 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644def7f-02d9-40d2-989f-5fde914f0c4f service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:45.306031 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644def7f-02d9-40d2-989f-5fde914f0c4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:45.316453 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0120b6-a344-4168-9761-34613fa3353b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:45.318392 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0120b6-a344-4168-9761-34613fa3353b service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:45.330166 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 913/1835] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:45 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:43:45.331274 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0120b6-a344-4168-9761-34613fa3353b service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:45.342223 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0120b6-a344-4168-9761-34613fa3353b service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:45.348177 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be84db79-9222-4bd7-a6b5-784c8041fad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:45.349087 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be84db79-9222-4bd7-a6b5-784c8041fad6 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:45.349872 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be84db79-9222-4bd7-a6b5-784c8041fad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:45.350304 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be84db79-9222-4bd7-a6b5-784c8041fad6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:45.351183 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be84db79-9222-4bd7-a6b5-784c8041fad6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:45.354181 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be84db79-9222-4bd7-a6b5-784c8041fad6 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:45.396664 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 914/1836] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:43:45 2022] POST /identity/v3/projects => generated 405 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:43:45.400626 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be84db79-9222-4bd7-a6b5-784c8041fad6 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:45.426536 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644def7f-02d9-40d2-989f-5fde914f0c4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:45.428398 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644def7f-02d9-40d2-989f-5fde914f0c4f service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:45.447125 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 923/1837] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:43:45.448229 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644def7f-02d9-40d2-989f-5fde914f0c4f service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:45.464736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644def7f-02d9-40d2-989f-5fde914f0c4f service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:45.480041 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:45.518159 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be84db79-9222-4bd7-a6b5-784c8041fad6 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:45.533477 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ea315-ce18-45ea-8b92-3b77e85847c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:45.537361 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ea315-ce18-45ea-8b92-3b77e85847c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:45.537657 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ea315-ce18-45ea-8b92-3b77e85847c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:45.537994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ea315-ce18-45ea-8b92-3b77e85847c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:45.538774 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002ea315-ce18-45ea-8b92-3b77e85847c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:45.543307 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002ea315-ce18-45ea-8b92-3b77e85847c8 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:45.572071 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 915/1838] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:43:45.572940 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ea315-ce18-45ea-8b92-3b77e85847c8 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:45.594148 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:45.619811 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b88b54-ff42-43a9-99f7-a3483ebd96dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:45.621340 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b88b54-ff42-43a9-99f7-a3483ebd96dc admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:45.621663 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b88b54-ff42-43a9-99f7-a3483ebd96dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:45.621952 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b88b54-ff42-43a9-99f7-a3483ebd96dc admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:45.622671 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b88b54-ff42-43a9-99f7-a3483ebd96dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:45.626309 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b88b54-ff42-43a9-99f7-a3483ebd96dc admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:45.659932 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 924/1839] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:43:45 2022] POST /identity/v3/users => generated 309 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:43:45.661187 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b88b54-ff42-43a9-99f7-a3483ebd96dc admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:45.676593 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b88b54-ff42-43a9-99f7-a3483ebd96dc admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:45.684562 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bfd448-98e6-4c7a-bb3a-c96317674db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:45.686319 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bfd448-98e6-4c7a-bb3a-c96317674db8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:45.688920 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bfd448-98e6-4c7a-bb3a-c96317674db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:45.689349 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bfd448-98e6-4c7a-bb3a-c96317674db8 admin admin] PATH_INFO: `/v3/projects/42cf277dca544b039b8c3dbfe5c8a0e8/users/9f1d0b15636243cfb7486fa13de03f49/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:45.701608 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ea315-ce18-45ea-8b92-3b77e85847c8 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:45.709991 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fbdb56-3571-4039-bd59-cc9782dd4ede service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:45.710987 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fbdb56-3571-4039-bd59-cc9782dd4ede service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:45.711507 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fbdb56-3571-4039-bd59-cc9782dd4ede service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:45.712161 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fbdb56-3571-4039-bd59-cc9782dd4ede service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:45.716581 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bfd448-98e6-4c7a-bb3a-c96317674db8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42cf277dca544b039b8c3dbfe5c8a0e8, user_id=9f1d0b15636243cfb7486fa13de03f49, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:45.722026 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bfd448-98e6-4c7a-bb3a-c96317674db8 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:45.746906 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 925/1840] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:43:45 2022] PUT /identity/v3/projects/42cf277dca544b039b8c3dbfe5c8a0e8/users/9f1d0b15636243cfb7486fa13de03f49/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:45.748030 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bfd448-98e6-4c7a-bb3a-c96317674db8 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:45.852097 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fbdb56-3571-4039-bd59-cc9782dd4ede service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:45.857437 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fbdb56-3571-4039-bd59-cc9782dd4ede service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:45.867201 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bfd448-98e6-4c7a-bb3a-c96317674db8 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:45.883402 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd8663b-8a79-4665-9564-9ed85e3ca6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:45.883402 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8663b-8a79-4665-9564-9ed85e3ca6d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:45.883402 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8663b-8a79-4665-9564-9ed85e3ca6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:45.883402 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8663b-8a79-4665-9564-9ed85e3ca6d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:45.883711 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd8663b-8a79-4665-9564-9ed85e3ca6d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:45.883711 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd8663b-8a79-4665-9564-9ed85e3ca6d0 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:45.899906 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 916/1841] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:45 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:43:45.901536 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fbdb56-3571-4039-bd59-cc9782dd4ede service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:45.929932 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 926/1842] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:43:45.946962 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fbdb56-3571-4039-bd59-cc9782dd4ede service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:45.950541 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd8663b-8a79-4665-9564-9ed85e3ca6d0 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:45.965962 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3113262b-fc28-4574-9ad8-f8f41c1958b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:45.967739 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3113262b-fc28-4574-9ad8-f8f41c1958b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:45.971096 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3113262b-fc28-4574-9ad8-f8f41c1958b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:45.971848 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3113262b-fc28-4574-9ad8-f8f41c1958b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:45.973722 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3113262b-fc28-4574-9ad8-f8f41c1958b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:45.980479 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3113262b-fc28-4574-9ad8-f8f41c1958b2 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:45.986659 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd8663b-8a79-4665-9564-9ed85e3ca6d0 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:45.996715 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7704a9f-a22b-447c-a2c2-5f46b02c916f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:45.997908 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7704a9f-a22b-447c-a2c2-5f46b02c916f admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:45.998428 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7704a9f-a22b-447c-a2c2-5f46b02c916f admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:45.998816 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7704a9f-a22b-447c-a2c2-5f46b02c916f admin admin] PATH_INFO: `/v3/projects/54151cd9d4ed49feb9f79594d20d9311/users/797a5993610d4bfc84263acf3d11956e/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:46.017620 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 917/1843] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:43:46.024071 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3113262b-fc28-4574-9ad8-f8f41c1958b2 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:46.051587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7704a9f-a22b-447c-a2c2-5f46b02c916f admin admin] RBAC: Authorizing `identity:create_grant(project_id=54151cd9d4ed49feb9f79594d20d9311, user_id=797a5993610d4bfc84263acf3d11956e, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:46.056416 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7704a9f-a22b-447c-a2c2-5f46b02c916f admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:46.060998 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3113262b-fc28-4574-9ad8-f8f41c1958b2 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:46.081054 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adb0369-c6dc-46da-bbb9-a7c063a641b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:46.081854 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb0369-c6dc-46da-bbb9-a7c063a641b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:46.082049 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb0369-c6dc-46da-bbb9-a7c063a641b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:46.082333 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb0369-c6dc-46da-bbb9-a7c063a641b3 admin admin] PATH_INFO: `/v3/projects/42cf277dca544b039b8c3dbfe5c8a0e8/users/9f1d0b15636243cfb7486fa13de03f49/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:46.086835 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 927/1844] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:43:45 2022] PUT /identity/v3/projects/54151cd9d4ed49feb9f79594d20d9311/users/797a5993610d4bfc84263acf3d11956e/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:46.089123 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7704a9f-a22b-447c-a2c2-5f46b02c916f admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:46.089288 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adb0369-c6dc-46da-bbb9-a7c063a641b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42cf277dca544b039b8c3dbfe5c8a0e8, user_id=9f1d0b15636243cfb7486fa13de03f49, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:46.098272 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adb0369-c6dc-46da-bbb9-a7c063a641b3 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:46.129263 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 918/1845] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:43:46 2022] PUT /identity/v3/projects/42cf277dca544b039b8c3dbfe5c8a0e8/users/9f1d0b15636243cfb7486fa13de03f49/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:46.136971 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb3b54-719d-4235-be13-f5ac04376c95 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:46.137561 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb3b54-719d-4235-be13-f5ac04376c95 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:46.138081 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb3b54-719d-4235-be13-f5ac04376c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:46.171928 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-40bb3b54-719d-4235-be13-f5ac04376c95 None None] MFA Rules not processed for user `9f1d0b15636243cfb7486fa13de03f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:46.173708 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-40bb3b54-719d-4235-be13-f5ac04376c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:46.253008 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:46.266060 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 919/1846] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:46 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:43:46.275143 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb3b54-719d-4235-be13-f5ac04376c95 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:46.313850 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7704a9f-a22b-447c-a2c2-5f46b02c916f admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:46.397481 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:46.425097 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb3b54-719d-4235-be13-f5ac04376c95 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:46.429743 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc0ad48-a8f8-4338-8e08-aef1b0c392a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:46.430462 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0ad48-a8f8-4338-8e08-aef1b0c392a2 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:46.430784 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0ad48-a8f8-4338-8e08-aef1b0c392a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:46.431076 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0ad48-a8f8-4338-8e08-aef1b0c392a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:46.443041 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc0ad48-a8f8-4338-8e08-aef1b0c392a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:46.444965 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc0ad48-a8f8-4338-8e08-aef1b0c392a2 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:46.480556 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 920/1847] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:46 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 16:43:46.490888 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b15d722-6da2-4d65-85cc-9da775af5ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:46.491847 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b15d722-6da2-4d65-85cc-9da775af5ecc admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:46.492279 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b15d722-6da2-4d65-85cc-9da775af5ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:46.492758 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b15d722-6da2-4d65-85cc-9da775af5ecc admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:46.494651 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b15d722-6da2-4d65-85cc-9da775af5ecc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:46.498250 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b15d722-6da2-4d65-85cc-9da775af5ecc admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:46.532094 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 928/1848] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 440 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:43:46.536752 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc0ad48-a8f8-4338-8e08-aef1b0c392a2 service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:46.554507 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc0ad48-a8f8-4338-8e08-aef1b0c392a2 service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:46.561692 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d7fd90-0075-4579-a94e-b62ecd2d81d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:46.566930 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7fd90-0075-4579-a94e-b62ecd2d81d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:46.567812 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7fd90-0075-4579-a94e-b62ecd2d81d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:46.568553 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7fd90-0075-4579-a94e-b62ecd2d81d0 admin admin] PATH_INFO: `/v3/projects/54151cd9d4ed49feb9f79594d20d9311/users/797a5993610d4bfc84263acf3d11956e/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:46.576242 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d7fd90-0075-4579-a94e-b62ecd2d81d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54151cd9d4ed49feb9f79594d20d9311, user_id=797a5993610d4bfc84263acf3d11956e, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:46.588512 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d7fd90-0075-4579-a94e-b62ecd2d81d0 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:46.621390 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b15d722-6da2-4d65-85cc-9da775af5ecc admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:46.638410 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b15d722-6da2-4d65-85cc-9da775af5ecc admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:46.650284 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 921/1849] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:43:46 2022] PUT /identity/v3/projects/54151cd9d4ed49feb9f79594d20d9311/users/797a5993610d4bfc84263acf3d11956e/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:46.662022 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e614ce3-68c3-4ba1-be11-7c3659884f8c None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:46.662313 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e614ce3-68c3-4ba1-be11-7c3659884f8c None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:46.662599 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e614ce3-68c3-4ba1-be11-7c3659884f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:46.693957 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-5e614ce3-68c3-4ba1-be11-7c3659884f8c None None] MFA Rules not processed for user `797a5993610d4bfc84263acf3d11956e`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:46.695272 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-5e614ce3-68c3-4ba1-be11-7c3659884f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:46.780190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4816e05-f131-4e8d-a14b-ddc5357162ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:46.780705 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4816e05-f131-4e8d-a14b-ddc5357162ba service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:46.780996 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4816e05-f131-4e8d-a14b-ddc5357162ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:46.781879 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4816e05-f131-4e8d-a14b-ddc5357162ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:46.810630 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 922/1850] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:46 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:46.823050 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e614ce3-68c3-4ba1-be11-7c3659884f8c None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:46.834711 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e614ce3-68c3-4ba1-be11-7c3659884f8c None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:46.840194 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ed9b9-fd83-4896-8e2a-93ed2d546870 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:46.841063 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ed9b9-fd83-4896-8e2a-93ed2d546870 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:46.841550 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ed9b9-fd83-4896-8e2a-93ed2d546870 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:46.841971 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ed9b9-fd83-4896-8e2a-93ed2d546870 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:46.853566 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604ed9b9-fd83-4896-8e2a-93ed2d546870 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:46.855771 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604ed9b9-fd83-4896-8e2a-93ed2d546870 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:46.868615 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 923/1851] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:46 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 16:43:46.873630 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4816e05-f131-4e8d-a14b-ddc5357162ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:46.875746 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4816e05-f131-4e8d-a14b-ddc5357162ba service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:46.887925 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 929/1852] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:43:47.106217 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ed9b9-fd83-4896-8e2a-93ed2d546870 service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:47.219411 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ed9b9-fd83-4896-8e2a-93ed2d546870 service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:47.243013 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78cbdb9-dc9b-455a-bce0-de3ce3303b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:47.247311 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78cbdb9-dc9b-455a-bce0-de3ce3303b1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:47.247865 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78cbdb9-dc9b-455a-bce0-de3ce3303b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:47.248375 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78cbdb9-dc9b-455a-bce0-de3ce3303b1b admin admin] PATH_INFO: `/v3/users/9ca86917a040495ea48905d363c9e8d7` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:47.251498 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78cbdb9-dc9b-455a-bce0-de3ce3303b1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ca86917a040495ea48905d363c9e8d7)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:47.255751 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78cbdb9-dc9b-455a-bce0-de3ce3303b1b admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:47.258602 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:47.369081 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-a78cbdb9-dc9b-455a-bce0-de3ce3303b1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ca86917a040495ea48905d363c9e8d7'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:47.371428 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-a78cbdb9-dc9b-455a-bce0-de3ce3303b1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ca86917a040495ea48905d363c9e8d7'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:47.400219 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-a78cbdb9-dc9b-455a-bce0-de3ce3303b1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ca86917a040495ea48905d363c9e8d7'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:47.419467 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-a78cbdb9-dc9b-455a-bce0-de3ce3303b1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ca86917a040495ea48905d363c9e8d7'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:47.420495 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-a78cbdb9-dc9b-455a-bce0-de3ce3303b1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ca86917a040495ea48905d363c9e8d7'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:47.437241 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 924/1853] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:43:47 2022] DELETE /identity/v3/users/9ca86917a040495ea48905d363c9e8d7 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:47.440008 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae85a896-855a-4ba4-88e5-102739a66085 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:47.440663 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae85a896-855a-4ba4-88e5-102739a66085 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:47.441033 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae85a896-855a-4ba4-88e5-102739a66085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:47.464931 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-ae85a896-855a-4ba4-88e5-102739a66085 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:47.467592 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-ae85a896-855a-4ba4-88e5-102739a66085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:47.553805 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 930/1854] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:47.563848 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78cbdb9-dc9b-455a-bce0-de3ce3303b1b admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:47.566213 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-a78cbdb9-dc9b-455a-bce0-de3ce3303b1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:47.659981 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78cbdb9-dc9b-455a-bce0-de3ce3303b1b admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:47.665293 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb205108-87a3-48d3-9849-463873326677 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:47.666038 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb205108-87a3-48d3-9849-463873326677 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:47.666534 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb205108-87a3-48d3-9849-463873326677 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:47.666918 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb205108-87a3-48d3-9849-463873326677 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:47.681765 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb205108-87a3-48d3-9849-463873326677 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:47.683290 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb205108-87a3-48d3-9849-463873326677 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:47.697568 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 925/1855] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:43:48.175213 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae85a896-855a-4ba4-88e5-102739a66085 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:48.176684 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-ae85a896-855a-4ba4-88e5-102739a66085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:48.191972 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae85a896-855a-4ba4-88e5-102739a66085 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:48.197773 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e97ff-d491-43c5-9cfe-d320f1c6ac3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:48.198755 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e97ff-d491-43c5-9cfe-d320f1c6ac3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:48.199046 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e97ff-d491-43c5-9cfe-d320f1c6ac3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:48.199778 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e97ff-d491-43c5-9cfe-d320f1c6ac3d admin admin] PATH_INFO: `/v3/projects/c2c5934d7e484e42972680eb65656f14` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:48.201033 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298e97ff-d491-43c5-9cfe-d320f1c6ac3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2c5934d7e484e42972680eb65656f14)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:48.203412 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298e97ff-d491-43c5-9cfe-d320f1c6ac3d admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:48.230626 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:48.261693 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-298e97ff-d491-43c5-9cfe-d320f1c6ac3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2c5934d7e484e42972680eb65656f14', 'request_id': 'req-298e97ff-d491-43c5-9cfe-d320f1c6ac3d'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:48.267667 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-298e97ff-d491-43c5-9cfe-d320f1c6ac3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2c5934d7e484e42972680eb65656f14', 'request_id': 'req-298e97ff-d491-43c5-9cfe-d320f1c6ac3d'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:48.274128 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-298e97ff-d491-43c5-9cfe-d320f1c6ac3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2c5934d7e484e42972680eb65656f14', 'request_id': 'req-298e97ff-d491-43c5-9cfe-d320f1c6ac3d'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:48.287336 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-298e97ff-d491-43c5-9cfe-d320f1c6ac3d admin admin] The token cache is being invalidate because project c2c5934d7e484e42972680eb65656f14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:43:48.287624 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-298e97ff-d491-43c5-9cfe-d320f1c6ac3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:48.288321 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-298e97ff-d491-43c5-9cfe-d320f1c6ac3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:48.289960 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 931/1856] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:43:48 2022] DELETE /identity/v3/projects/c2c5934d7e484e42972680eb65656f14 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:48.295507 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb205108-87a3-48d3-9849-463873326677 service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:48.305801 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb205108-87a3-48d3-9849-463873326677 service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:48.365042 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511e183d-0f8e-48a4-92e1-d38c362a5d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:48.365947 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e183d-0f8e-48a4-92e1-d38c362a5d82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:48.366229 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e183d-0f8e-48a4-92e1-d38c362a5d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:48.366504 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e183d-0f8e-48a4-92e1-d38c362a5d82 admin admin] PATH_INFO: `/v3/users/797a5993610d4bfc84263acf3d11956e` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:48.367707 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511e183d-0f8e-48a4-92e1-d38c362a5d82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=797a5993610d4bfc84263acf3d11956e)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:48.369820 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511e183d-0f8e-48a4-92e1-d38c362a5d82 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:48.394171 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:48.440488 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-511e183d-0f8e-48a4-92e1-d38c362a5d82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '797a5993610d4bfc84263acf3d11956e'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:48.441061 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-511e183d-0f8e-48a4-92e1-d38c362a5d82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '797a5993610d4bfc84263acf3d11956e'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:48.450909 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-511e183d-0f8e-48a4-92e1-d38c362a5d82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '797a5993610d4bfc84263acf3d11956e'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:48.478375 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-511e183d-0f8e-48a4-92e1-d38c362a5d82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '797a5993610d4bfc84263acf3d11956e'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:48.479400 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-511e183d-0f8e-48a4-92e1-d38c362a5d82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '797a5993610d4bfc84263acf3d11956e'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:48.508916 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 926/1857] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:43:48 2022] DELETE /identity/v3/users/797a5993610d4bfc84263acf3d11956e => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:48.881521 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e97ff-d491-43c5-9cfe-d320f1c6ac3d admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:48.883213 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-298e97ff-d491-43c5-9cfe-d320f1c6ac3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:48.914371 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e97ff-d491-43c5-9cfe-d320f1c6ac3d admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:49.017429 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee8c003-e658-4a62-b151-f4c565b0e4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:49.019199 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8c003-e658-4a62-b151-f4c565b0e4ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:49.019493 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8c003-e658-4a62-b151-f4c565b0e4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:49.019922 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8c003-e658-4a62-b151-f4c565b0e4ae admin admin] PATH_INFO: `/v3/projects/54151cd9d4ed49feb9f79594d20d9311` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:49.022623 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee8c003-e658-4a62-b151-f4c565b0e4ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=54151cd9d4ed49feb9f79594d20d9311)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:49.033997 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee8c003-e658-4a62-b151-f4c565b0e4ae admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:49.153093 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-bee8c003-e658-4a62-b151-f4c565b0e4ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54151cd9d4ed49feb9f79594d20d9311', 'request_id': 'req-bee8c003-e658-4a62-b151-f4c565b0e4ae'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:49.167152 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-bee8c003-e658-4a62-b151-f4c565b0e4ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54151cd9d4ed49feb9f79594d20d9311', 'request_id': 'req-bee8c003-e658-4a62-b151-f4c565b0e4ae'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:49.180536 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-bee8c003-e658-4a62-b151-f4c565b0e4ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54151cd9d4ed49feb9f79594d20d9311', 'request_id': 'req-bee8c003-e658-4a62-b151-f4c565b0e4ae'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:49.226935 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-bee8c003-e658-4a62-b151-f4c565b0e4ae admin admin] The token cache is being invalidate because project 54151cd9d4ed49feb9f79594d20d9311 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:43:49.227732 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-bee8c003-e658-4a62-b151-f4c565b0e4ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:49.228851 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-bee8c003-e658-4a62-b151-f4c565b0e4ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:49.231530 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 932/1858] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:43:48 2022] DELETE /identity/v3/projects/54151cd9d4ed49feb9f79594d20d9311 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:49.236374 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48496830-4c8e-4ac3-bc60-66a78368c12f None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:49.236674 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48496830-4c8e-4ac3-bc60-66a78368c12f None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:49.236969 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48496830-4c8e-4ac3-bc60-66a78368c12f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:49.276124 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-48496830-4c8e-4ac3-bc60-66a78368c12f None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:49.277537 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-48496830-4c8e-4ac3-bc60-66a78368c12f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:49.395070 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 927/1859] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:49.402566 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e327d-68d2-4b76-b549-45998974d62a None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:49.403221 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e327d-68d2-4b76-b549-45998974d62a None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:49.403940 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e327d-68d2-4b76-b549-45998974d62a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:49.430507 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-be7e327d-68d2-4b76-b549-45998974d62a None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:49.431958 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-be7e327d-68d2-4b76-b549-45998974d62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:49.451550 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 933/1860] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:43:49.452306 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48496830-4c8e-4ac3-bc60-66a78368c12f None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:49.464516 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48496830-4c8e-4ac3-bc60-66a78368c12f None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:49.486696 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef9755e-8741-4ee1-a011-126c3a346527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:49.487488 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef9755e-8741-4ee1-a011-126c3a346527 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:49.487860 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef9755e-8741-4ee1-a011-126c3a346527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:49.488230 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef9755e-8741-4ee1-a011-126c3a346527 admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:49.489067 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef9755e-8741-4ee1-a011-126c3a346527 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:43:49.489494 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef9755e-8741-4ee1-a011-126c3a346527 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:49.491206 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef9755e-8741-4ee1-a011-126c3a346527 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:49.512138 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 928/1861] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:43:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:43:49.513003 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e327d-68d2-4b76-b549-45998974d62a None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:49.529341 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e327d-68d2-4b76-b549-45998974d62a None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:49.533914 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8436837-be94-47f4-b5b2-2f8a404726e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:49.534675 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8436837-be94-47f4-b5b2-2f8a404726e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:49.534974 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8436837-be94-47f4-b5b2-2f8a404726e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:49.535260 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8436837-be94-47f4-b5b2-2f8a404726e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:49.536051 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8436837-be94-47f4-b5b2-2f8a404726e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:49.538051 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8436837-be94-47f4-b5b2-2f8a404726e3 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:49.580822 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 934/1862] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:43:49 2022] POST /identity/v3/projects => generated 377 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:43:49.582511 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef9755e-8741-4ee1-a011-126c3a346527 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:49.595301 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef9755e-8741-4ee1-a011-126c3a346527 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:49.741136 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab518e0b-a602-4e5f-9032-0631001b95e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:49.742083 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab518e0b-a602-4e5f-9032-0631001b95e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:49.743541 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab518e0b-a602-4e5f-9032-0631001b95e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:49.743948 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab518e0b-a602-4e5f-9032-0631001b95e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:49.745197 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab518e0b-a602-4e5f-9032-0631001b95e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:49.759694 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab518e0b-a602-4e5f-9032-0631001b95e2 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:49.790823 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 929/1863] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:43:49 2022] POST /identity/v3/users => generated 289 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:43:49.797958 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8436837-be94-47f4-b5b2-2f8a404726e3 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:49.813110 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8436837-be94-47f4-b5b2-2f8a404726e3 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:49.824046 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542c3d2c-8e22-416d-8640-f8e3bfb10b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:49.825212 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c3d2c-8e22-416d-8640-f8e3bfb10b92 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:49.825619 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c3d2c-8e22-416d-8640-f8e3bfb10b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:49.825901 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c3d2c-8e22-416d-8640-f8e3bfb10b92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:49.826808 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542c3d2c-8e22-416d-8640-f8e3bfb10b92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:49.828705 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542c3d2c-8e22-416d-8640-f8e3bfb10b92 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:49.831883 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:49.857423 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 935/1864] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:49 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:43:49.861724 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab518e0b-a602-4e5f-9032-0631001b95e2 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:49.873928 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab518e0b-a602-4e5f-9032-0631001b95e2 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:49.881551 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd0c76a-c3ee-4f20-8b7f-29c71c645630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:49.882615 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0c76a-c3ee-4f20-8b7f-29c71c645630 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:49.884645 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0c76a-c3ee-4f20-8b7f-29c71c645630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:49.885340 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0c76a-c3ee-4f20-8b7f-29c71c645630 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:49.886425 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd0c76a-c3ee-4f20-8b7f-29c71c645630 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:49.895091 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd0c76a-c3ee-4f20-8b7f-29c71c645630 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:49.915852 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 930/1865] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:49 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:43:49.918500 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542c3d2c-8e22-416d-8640-f8e3bfb10b92 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:49.929802 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542c3d2c-8e22-416d-8640-f8e3bfb10b92 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:49.934850 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad46cab5-da4a-43f1-b309-4a86025a66d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:49.935747 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46cab5-da4a-43f1-b309-4a86025a66d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:49.936034 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46cab5-da4a-43f1-b309-4a86025a66d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:49.936314 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46cab5-da4a-43f1-b309-4a86025a66d7 admin admin] PATH_INFO: `/v3/projects/fd67d9207fc64ec29ea1bb9cf008c0ec/users/3369ab1f03f44454a344c225ec66c1af/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:49.952657 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad46cab5-da4a-43f1-b309-4a86025a66d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd67d9207fc64ec29ea1bb9cf008c0ec, user_id=3369ab1f03f44454a344c225ec66c1af, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:49.955458 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad46cab5-da4a-43f1-b309-4a86025a66d7 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:49.974768 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 936/1866] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:43:49 2022] PUT /identity/v3/projects/fd67d9207fc64ec29ea1bb9cf008c0ec/users/3369ab1f03f44454a344c225ec66c1af/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:49.983134 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6cffe3-89e3-49ea-8a71-31a3193a2930 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:49.983593 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6cffe3-89e3-49ea-8a71-31a3193a2930 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:49.983884 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6cffe3-89e3-49ea-8a71-31a3193a2930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:49.999527 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:50.021901 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-7c6cffe3-89e3-49ea-8a71-31a3193a2930 None None] MFA Rules not processed for user `3369ab1f03f44454a344c225ec66c1af`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:50.023153 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-7c6cffe3-89e3-49ea-8a71-31a3193a2930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:50.089890 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 931/1867] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:43:50.095097 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad46cab5-da4a-43f1-b309-4a86025a66d7 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:50.157432 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad46cab5-da4a-43f1-b309-4a86025a66d7 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:50.164169 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab8003-1a28-4864-800e-55ab60857762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:50.164849 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab8003-1a28-4864-800e-55ab60857762 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:50.165140 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab8003-1a28-4864-800e-55ab60857762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:50.165689 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab8003-1a28-4864-800e-55ab60857762 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:50.166223 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cab8003-1a28-4864-800e-55ab60857762 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:50.168308 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cab8003-1a28-4864-800e-55ab60857762 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:50.203119 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 937/1868] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:43:50 2022] POST /identity/v3/projects => generated 377 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:43:50.206907 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6cffe3-89e3-49ea-8a71-31a3193a2930 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:50.218270 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6cffe3-89e3-49ea-8a71-31a3193a2930 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:50.281692 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4585b7-c49c-4346-a17e-fc0062f80652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:50.282397 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4585b7-c49c-4346-a17e-fc0062f80652 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:50.282686 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4585b7-c49c-4346-a17e-fc0062f80652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:50.283005 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4585b7-c49c-4346-a17e-fc0062f80652 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:50.283711 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4585b7-c49c-4346-a17e-fc0062f80652 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:50.286708 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4585b7-c49c-4346-a17e-fc0062f80652 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:50.308336 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab8003-1a28-4864-800e-55ab60857762 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:50.308708 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 932/1869] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:43:50 2022] POST /identity/v3/users => generated 295 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:43:50.310542 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-7cab8003-1a28-4864-800e-55ab60857762 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:50.312851 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4585b7-c49c-4346-a17e-fc0062f80652 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:50.324181 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4585b7-c49c-4346-a17e-fc0062f80652 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:50.329458 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc19eea-f7d7-43ca-a115-a108744feaf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:50.330119 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc19eea-f7d7-43ca-a115-a108744feaf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:50.330403 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc19eea-f7d7-43ca-a115-a108744feaf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:50.330793 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc19eea-f7d7-43ca-a115-a108744feaf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:50.331654 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc19eea-f7d7-43ca-a115-a108744feaf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:50.333278 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc19eea-f7d7-43ca-a115-a108744feaf4 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:50.343215 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 933/1870] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:43:50.348079 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc19eea-f7d7-43ca-a115-a108744feaf4 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:50.358265 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc19eea-f7d7-43ca-a115-a108744feaf4 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:50.363961 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef5171-8507-4178-9003-d18e8af7ea28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:50.364843 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef5171-8507-4178-9003-d18e8af7ea28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:50.365135 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef5171-8507-4178-9003-d18e8af7ea28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:50.365434 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef5171-8507-4178-9003-d18e8af7ea28 admin admin] PATH_INFO: `/v3/projects/c53d547411554dd69679ad3935bb7510/users/018effefcef14d9ca86e200666cbc2e5/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:50.374017 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab8003-1a28-4864-800e-55ab60857762 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:50.377697 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3bc63d-37a2-4995-a368-b5ef5f3311ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:50.378352 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3bc63d-37a2-4995-a368-b5ef5f3311ea service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:50.378641 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3bc63d-37a2-4995-a368-b5ef5f3311ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:50.378928 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3bc63d-37a2-4995-a368-b5ef5f3311ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:50.380712 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-4b3bc63d-37a2-4995-a368-b5ef5f3311ea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:50.382111 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ef5171-8507-4178-9003-d18e8af7ea28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c53d547411554dd69679ad3935bb7510, user_id=018effefcef14d9ca86e200666cbc2e5, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:50.385305 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ef5171-8507-4178-9003-d18e8af7ea28 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:50.399811 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 934/1871] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:43:50 2022] PUT /identity/v3/projects/c53d547411554dd69679ad3935bb7510/users/018effefcef14d9ca86e200666cbc2e5/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:43:50.403864 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef5171-8507-4178-9003-d18e8af7ea28 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:50.449084 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3bc63d-37a2-4995-a368-b5ef5f3311ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:50.450597 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3bc63d-37a2-4995-a368-b5ef5f3311ea service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:50.461414 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 938/1872] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:50 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 16:43:50.466239 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:50.479990 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef5171-8507-4178-9003-d18e8af7ea28 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:50.485096 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334c291a-7c33-4dfb-a204-202c7140d629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:50.485809 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c291a-7c33-4dfb-a204-202c7140d629 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:50.486124 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c291a-7c33-4dfb-a204-202c7140d629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:50.486386 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c291a-7c33-4dfb-a204-202c7140d629 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:50.487535 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334c291a-7c33-4dfb-a204-202c7140d629 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:50.488762 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334c291a-7c33-4dfb-a204-202c7140d629 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:50.499095 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 935/1873] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:50 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:43:50.503507 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3bc63d-37a2-4995-a368-b5ef5f3311ea service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:50.514586 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3bc63d-37a2-4995-a368-b5ef5f3311ea service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:50.519863 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a0ad2-3dfe-4b68-b846-6f04e4cb7e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:50.520786 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a0ad2-3dfe-4b68-b846-6f04e4cb7e97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:50.521075 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a0ad2-3dfe-4b68-b846-6f04e4cb7e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:50.521398 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a0ad2-3dfe-4b68-b846-6f04e4cb7e97 admin admin] PATH_INFO: `/v3/projects/c53d547411554dd69679ad3935bb7510/users/018effefcef14d9ca86e200666cbc2e5/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:50.523529 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256a0ad2-3dfe-4b68-b846-6f04e4cb7e97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c53d547411554dd69679ad3935bb7510, user_id=018effefcef14d9ca86e200666cbc2e5, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:50.526330 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256a0ad2-3dfe-4b68-b846-6f04e4cb7e97 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:50.529814 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:50.543194 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 939/1874] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:43:50 2022] PUT /identity/v3/projects/c53d547411554dd69679ad3935bb7510/users/018effefcef14d9ca86e200666cbc2e5/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:50.553981 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180fe52a-a0e2-46fe-aec6-528ac4b4afa8 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:50.554289 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180fe52a-a0e2-46fe-aec6-528ac4b4afa8 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:50.554605 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180fe52a-a0e2-46fe-aec6-528ac4b4afa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:50.583292 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-180fe52a-a0e2-46fe-aec6-528ac4b4afa8 None None] MFA Rules not processed for user `018effefcef14d9ca86e200666cbc2e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:50.584525 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-180fe52a-a0e2-46fe-aec6-528ac4b4afa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:50.666439 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 936/1875] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:50 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:50.677207 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a0ad2-3dfe-4b68-b846-6f04e4cb7e97 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:50.679203 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-256a0ad2-3dfe-4b68-b846-6f04e4cb7e97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:50.749581 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a0ad2-3dfe-4b68-b846-6f04e4cb7e97 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:50.753899 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b474013d-3378-4a89-a0c2-2301558caefa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:50.754562 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b474013d-3378-4a89-a0c2-2301558caefa service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:50.754852 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b474013d-3378-4a89-a0c2-2301558caefa service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:50.755136 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b474013d-3378-4a89-a0c2-2301558caefa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:50.822268 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b474013d-3378-4a89-a0c2-2301558caefa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:50.823978 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b474013d-3378-4a89-a0c2-2301558caefa service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:50.834387 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 940/1876] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:50 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:43:51.355736 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180fe52a-a0e2-46fe-aec6-528ac4b4afa8 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:51.357809 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-180fe52a-a0e2-46fe-aec6-528ac4b4afa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:51.370635 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b474013d-3378-4a89-a0c2-2301558caefa service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:51.373166 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b474013d-3378-4a89-a0c2-2301558caefa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:51.446123 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180fe52a-a0e2-46fe-aec6-528ac4b4afa8 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:51.451235 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfe2367-a43b-49dd-98f0-5d7d34121b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:51.452345 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe2367-a43b-49dd-98f0-5d7d34121b50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:51.452634 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe2367-a43b-49dd-98f0-5d7d34121b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:51.452968 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe2367-a43b-49dd-98f0-5d7d34121b50 admin admin] PATH_INFO: `/v3/users/913b81ca38a6430a95597d70ebe42a0a` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:51.454841 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfe2367-a43b-49dd-98f0-5d7d34121b50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=913b81ca38a6430a95597d70ebe42a0a)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:51.454970 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b474013d-3378-4a89-a0c2-2301558caefa service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:51.456970 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfe2367-a43b-49dd-98f0-5d7d34121b50 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:51.460074 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f5775-9385-42ce-8054-3e90185702cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:51.460961 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f5775-9385-42ce-8054-3e90185702cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:51.461714 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f5775-9385-42ce-8054-3e90185702cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:51.462007 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f5775-9385-42ce-8054-3e90185702cc admin admin] PATH_INFO: `/v3/users/eeddf9c1eea74b33b7e23f322ceee9e6` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:51.463204 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886f5775-9385-42ce-8054-3e90185702cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=eeddf9c1eea74b33b7e23f322ceee9e6)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:51.465125 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886f5775-9385-42ce-8054-3e90185702cc admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:51.506527 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-fcfe2367-a43b-49dd-98f0-5d7d34121b50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '913b81ca38a6430a95597d70ebe42a0a'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:51.507127 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-fcfe2367-a43b-49dd-98f0-5d7d34121b50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '913b81ca38a6430a95597d70ebe42a0a'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:51.515024 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-fcfe2367-a43b-49dd-98f0-5d7d34121b50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '913b81ca38a6430a95597d70ebe42a0a'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:51.517491 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-886f5775-9385-42ce-8054-3e90185702cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eeddf9c1eea74b33b7e23f322ceee9e6'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:51.518146 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-886f5775-9385-42ce-8054-3e90185702cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eeddf9c1eea74b33b7e23f322ceee9e6'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:51.518752 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-886f5775-9385-42ce-8054-3e90185702cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eeddf9c1eea74b33b7e23f322ceee9e6'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:51.523299 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:51.527789 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-886f5775-9385-42ce-8054-3e90185702cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eeddf9c1eea74b33b7e23f322ceee9e6'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:51.528567 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-fcfe2367-a43b-49dd-98f0-5d7d34121b50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '913b81ca38a6430a95597d70ebe42a0a'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:51.529454 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-fcfe2367-a43b-49dd-98f0-5d7d34121b50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '913b81ca38a6430a95597d70ebe42a0a'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:51.533972 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:51.547748 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-886f5775-9385-42ce-8054-3e90185702cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eeddf9c1eea74b33b7e23f322ceee9e6'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:51.547944 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 937/1877] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:43:51 2022] DELETE /identity/v3/users/913b81ca38a6430a95597d70ebe42a0a => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:51.552245 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c893eea-1bb8-4102-8a41-3b633c007c6d None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:51.552539 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c893eea-1bb8-4102-8a41-3b633c007c6d None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:51.552827 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c893eea-1bb8-4102-8a41-3b633c007c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:51.560908 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 941/1878] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:43:51 2022] DELETE /identity/v3/users/eeddf9c1eea74b33b7e23f322ceee9e6 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:43:51.565764 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e2734-462a-4160-8a39-1cc1b94204d0 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:51.566065 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e2734-462a-4160-8a39-1cc1b94204d0 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:51.566351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e2734-462a-4160-8a39-1cc1b94204d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:51.571061 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-7c893eea-1bb8-4102-8a41-3b633c007c6d None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:51.572283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-7c893eea-1bb8-4102-8a41-3b633c007c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:51.583459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-577e2734-462a-4160-8a39-1cc1b94204d0 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:51.584654 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-577e2734-462a-4160-8a39-1cc1b94204d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:51.639074 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 938/1879] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:51.649438 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c893eea-1bb8-4102-8a41-3b633c007c6d None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:51.650844 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-7c893eea-1bb8-4102-8a41-3b633c007c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:51.652406 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 942/1880] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:43:51.663128 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577e2734-462a-4160-8a39-1cc1b94204d0 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:51.664531 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-577e2734-462a-4160-8a39-1cc1b94204d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:51.710801 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c893eea-1bb8-4102-8a41-3b633c007c6d None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:51.715417 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c611327-a72b-43d3-80d2-d04c8bd2ccb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:51.716143 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c611327-a72b-43d3-80d2-d04c8bd2ccb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:51.716497 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c611327-a72b-43d3-80d2-d04c8bd2ccb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:51.716843 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c611327-a72b-43d3-80d2-d04c8bd2ccb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:51.728188 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c611327-a72b-43d3-80d2-d04c8bd2ccb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:51.728748 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577e2734-462a-4160-8a39-1cc1b94204d0 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:51.729698 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c611327-a72b-43d3-80d2-d04c8bd2ccb0 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:51.733450 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7233c8ca-3dad-4110-9f0f-2d060e39b860 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:51.734943 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7233c8ca-3dad-4110-9f0f-2d060e39b860 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:51.735226 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7233c8ca-3dad-4110-9f0f-2d060e39b860 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:51.735499 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7233c8ca-3dad-4110-9f0f-2d060e39b860 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:51.741804 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 939/1881] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 16:43:51.746676 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7233c8ca-3dad-4110-9f0f-2d060e39b860 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:51.748369 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7233c8ca-3dad-4110-9f0f-2d060e39b860 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:51.759841 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 943/1882] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 16:43:52.374473 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c611327-a72b-43d3-80d2-d04c8bd2ccb0 service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:52.376007 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-5c611327-a72b-43d3-80d2-d04c8bd2ccb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:52.390076 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c611327-a72b-43d3-80d2-d04c8bd2ccb0 service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:52.396936 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c893cee-f07d-4325-9a43-dd4cadda9d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:52.398167 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c893cee-f07d-4325-9a43-dd4cadda9d38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:52.398757 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c893cee-f07d-4325-9a43-dd4cadda9d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:52.399254 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c893cee-f07d-4325-9a43-dd4cadda9d38 admin admin] PATH_INFO: `/v3/projects/7b4343c7f07f40e0a20bdd3c6720a757` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:52.401033 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c893cee-f07d-4325-9a43-dd4cadda9d38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b4343c7f07f40e0a20bdd3c6720a757)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:52.403503 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c893cee-f07d-4325-9a43-dd4cadda9d38 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:52.496403 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-2c893cee-f07d-4325-9a43-dd4cadda9d38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b4343c7f07f40e0a20bdd3c6720a757', 'request_id': 'req-2c893cee-f07d-4325-9a43-dd4cadda9d38'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:52.506875 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-2c893cee-f07d-4325-9a43-dd4cadda9d38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b4343c7f07f40e0a20bdd3c6720a757', 'request_id': 'req-2c893cee-f07d-4325-9a43-dd4cadda9d38'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:52.519177 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-2c893cee-f07d-4325-9a43-dd4cadda9d38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b4343c7f07f40e0a20bdd3c6720a757', 'request_id': 'req-2c893cee-f07d-4325-9a43-dd4cadda9d38'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:52.530640 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7233c8ca-3dad-4110-9f0f-2d060e39b860 service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:52.532771 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-7233c8ca-3dad-4110-9f0f-2d060e39b860 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:52.551673 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7233c8ca-3dad-4110-9f0f-2d060e39b860 service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:52.553392 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-2c893cee-f07d-4325-9a43-dd4cadda9d38 admin admin] The token cache is being invalidate because project 7b4343c7f07f40e0a20bdd3c6720a757 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:43:52.553697 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-2c893cee-f07d-4325-9a43-dd4cadda9d38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:52.554463 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-2c893cee-f07d-4325-9a43-dd4cadda9d38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:52.557347 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 940/1883] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:43:52 2022] DELETE /identity/v3/projects/7b4343c7f07f40e0a20bdd3c6720a757 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:52.562384 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c893cee-f07d-4325-9a43-dd4cadda9d38 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:52.565928 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-2c893cee-f07d-4325-9a43-dd4cadda9d38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:52.609665 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:52.692858 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c893cee-f07d-4325-9a43-dd4cadda9d38 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:52.697365 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3372e37-61f1-489c-af74-9ed810d46469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:52.698588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3372e37-61f1-489c-af74-9ed810d46469 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:52.698872 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3372e37-61f1-489c-af74-9ed810d46469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:52.699235 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3372e37-61f1-489c-af74-9ed810d46469 admin admin] PATH_INFO: `/v3/projects/217bf21d2bda40d99a25b20793f0e835` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:52.700420 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3372e37-61f1-489c-af74-9ed810d46469 admin admin] RBAC: Authorizing `identity:delete_project(project_id=217bf21d2bda40d99a25b20793f0e835)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:52.702535 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3372e37-61f1-489c-af74-9ed810d46469 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:52.704149 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba47810-28a6-4de8-965e-66238acab19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:52.705182 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba47810-28a6-4de8-965e-66238acab19b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:52.705479 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba47810-28a6-4de8-965e-66238acab19b admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:52.705812 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba47810-28a6-4de8-965e-66238acab19b admin admin] PATH_INFO: `/v3/users/9f1d0b15636243cfb7486fa13de03f49` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:52.711680 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba47810-28a6-4de8-965e-66238acab19b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f1d0b15636243cfb7486fa13de03f49)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:52.715962 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba47810-28a6-4de8-965e-66238acab19b admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:52.729515 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:52.774667 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-b3372e37-61f1-489c-af74-9ed810d46469 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '217bf21d2bda40d99a25b20793f0e835', 'request_id': 'req-b3372e37-61f1-489c-af74-9ed810d46469'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:52.781638 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-b3372e37-61f1-489c-af74-9ed810d46469 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '217bf21d2bda40d99a25b20793f0e835', 'request_id': 'req-b3372e37-61f1-489c-af74-9ed810d46469'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:52.783409 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-2ba47810-28a6-4de8-965e-66238acab19b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f1d0b15636243cfb7486fa13de03f49'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:52.784111 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-2ba47810-28a6-4de8-965e-66238acab19b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f1d0b15636243cfb7486fa13de03f49'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:52.787952 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-b3372e37-61f1-489c-af74-9ed810d46469 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '217bf21d2bda40d99a25b20793f0e835', 'request_id': 'req-b3372e37-61f1-489c-af74-9ed810d46469'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:52.794136 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-2ba47810-28a6-4de8-965e-66238acab19b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f1d0b15636243cfb7486fa13de03f49'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:52.804048 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-b3372e37-61f1-489c-af74-9ed810d46469 admin admin] The token cache is being invalidate because project 217bf21d2bda40d99a25b20793f0e835 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:43:52.804607 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-b3372e37-61f1-489c-af74-9ed810d46469 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:52.805423 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-b3372e37-61f1-489c-af74-9ed810d46469 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:52.807556 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-2ba47810-28a6-4de8-965e-66238acab19b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f1d0b15636243cfb7486fa13de03f49'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:52.808066 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 944/1884] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:43:52 2022] DELETE /identity/v3/projects/217bf21d2bda40d99a25b20793f0e835 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:43:52.811322 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-2ba47810-28a6-4de8-965e-66238acab19b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f1d0b15636243cfb7486fa13de03f49'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:52.811761 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3372e37-61f1-489c-af74-9ed810d46469 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:52.813423 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b3372e37-61f1-489c-af74-9ed810d46469 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:52.827217 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 941/1885] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:43:52 2022] DELETE /identity/v3/users/9f1d0b15636243cfb7486fa13de03f49 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:52.899545 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3372e37-61f1-489c-af74-9ed810d46469 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:52.968321 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b97c80-4cec-416c-9467-c5e85d73e0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:52.969329 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b97c80-4cec-416c-9467-c5e85d73e0cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:52.969621 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b97c80-4cec-416c-9467-c5e85d73e0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:52.969919 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b97c80-4cec-416c-9467-c5e85d73e0cc admin admin] PATH_INFO: `/v3/users/4c77abae271e49908a89fc66747843d9` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:52.971179 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b97c80-4cec-416c-9467-c5e85d73e0cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c77abae271e49908a89fc66747843d9)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:52.973174 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b97c80-4cec-416c-9467-c5e85d73e0cc admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:53.042650 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-c4b97c80-4cec-416c-9467-c5e85d73e0cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c77abae271e49908a89fc66747843d9'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:53.043344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-c4b97c80-4cec-416c-9467-c5e85d73e0cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c77abae271e49908a89fc66747843d9'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:53.044354 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-c4b97c80-4cec-416c-9467-c5e85d73e0cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c77abae271e49908a89fc66747843d9'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:53.049634 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:53.054005 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-c4b97c80-4cec-416c-9467-c5e85d73e0cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c77abae271e49908a89fc66747843d9'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:53.076261 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-c4b97c80-4cec-416c-9467-c5e85d73e0cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c77abae271e49908a89fc66747843d9'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:53.092989 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 945/1886] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:43:52 2022] DELETE /identity/v3/users/4c77abae271e49908a89fc66747843d9 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:43:53.166834 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba47810-28a6-4de8-965e-66238acab19b admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:53.168456 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-2ba47810-28a6-4de8-965e-66238acab19b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:53.182563 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba47810-28a6-4de8-965e-66238acab19b admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:53.252408 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572088e5-417f-47f2-880f-89d56a1ae6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:53.253410 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572088e5-417f-47f2-880f-89d56a1ae6ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:53.253697 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572088e5-417f-47f2-880f-89d56a1ae6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:53.253975 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572088e5-417f-47f2-880f-89d56a1ae6ec admin admin] PATH_INFO: `/v3/projects/42cf277dca544b039b8c3dbfe5c8a0e8` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:53.255163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572088e5-417f-47f2-880f-89d56a1ae6ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=42cf277dca544b039b8c3dbfe5c8a0e8)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:53.257362 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572088e5-417f-47f2-880f-89d56a1ae6ec admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:53.312476 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:53.321489 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-572088e5-417f-47f2-880f-89d56a1ae6ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42cf277dca544b039b8c3dbfe5c8a0e8', 'request_id': 'req-572088e5-417f-47f2-880f-89d56a1ae6ec'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:53.330434 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-572088e5-417f-47f2-880f-89d56a1ae6ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42cf277dca544b039b8c3dbfe5c8a0e8', 'request_id': 'req-572088e5-417f-47f2-880f-89d56a1ae6ec'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:53.338718 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-572088e5-417f-47f2-880f-89d56a1ae6ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42cf277dca544b039b8c3dbfe5c8a0e8', 'request_id': 'req-572088e5-417f-47f2-880f-89d56a1ae6ec'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:53.353973 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-572088e5-417f-47f2-880f-89d56a1ae6ec admin admin] The token cache is being invalidate because project 42cf277dca544b039b8c3dbfe5c8a0e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:43:53.354372 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-572088e5-417f-47f2-880f-89d56a1ae6ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:53.355135 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-572088e5-417f-47f2-880f-89d56a1ae6ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:53.357240 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 942/1887] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:43:53 2022] DELETE /identity/v3/projects/42cf277dca544b039b8c3dbfe5c8a0e8 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:53.362770 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668291c4-0af1-425f-85d7-dabd74897eb8 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:53.363136 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668291c4-0af1-425f-85d7-dabd74897eb8 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:53.363850 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668291c4-0af1-425f-85d7-dabd74897eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:53.401483 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-668291c4-0af1-425f-85d7-dabd74897eb8 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:53.402510 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-668291c4-0af1-425f-85d7-dabd74897eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:53.436063 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572088e5-417f-47f2-880f-89d56a1ae6ec admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:53.437589 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-572088e5-417f-47f2-880f-89d56a1ae6ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:53.485294 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 946/1888] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:43:53.492914 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71758364-0903-45df-9574-961b352ec335 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:53.493247 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71758364-0903-45df-9574-961b352ec335 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:53.493534 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71758364-0903-45df-9574-961b352ec335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:53.514617 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-71758364-0903-45df-9574-961b352ec335 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:53.516401 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-71758364-0903-45df-9574-961b352ec335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:53.519711 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572088e5-417f-47f2-880f-89d56a1ae6ec admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:53.525187 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 947/1889] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:43:53.528701 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aa1551-89ce-4e6d-a861-a50d48c0ac8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:53.529401 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71758364-0903-45df-9574-961b352ec335 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:53.529775 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa1551-89ce-4e6d-a861-a50d48c0ac8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:53.530134 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa1551-89ce-4e6d-a861-a50d48c0ac8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:53.530501 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa1551-89ce-4e6d-a861-a50d48c0ac8a admin admin] PATH_INFO: `/v3/projects/31b13cf1a01f425abb877ba397378765` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:53.532422 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aa1551-89ce-4e6d-a861-a50d48c0ac8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=31b13cf1a01f425abb877ba397378765)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:53.534474 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aa1551-89ce-4e6d-a861-a50d48c0ac8a admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:53.540828 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71758364-0903-45df-9574-961b352ec335 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:53.548186 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c18e86-b35e-46d3-be65-df2d26e481d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:53.548820 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c18e86-b35e-46d3-be65-df2d26e481d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:53.549106 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c18e86-b35e-46d3-be65-df2d26e481d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:53.549413 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c18e86-b35e-46d3-be65-df2d26e481d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:53.550155 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c18e86-b35e-46d3-be65-df2d26e481d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:43:53.550463 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c18e86-b35e-46d3-be65-df2d26e481d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:53.552255 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c18e86-b35e-46d3-be65-df2d26e481d3 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:53.565211 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 948/1890] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:43:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:43:53.569555 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c18e86-b35e-46d3-be65-df2d26e481d3 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:53.581974 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c18e86-b35e-46d3-be65-df2d26e481d3 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:53.596570 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-c1aa1551-89ce-4e6d-a861-a50d48c0ac8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31b13cf1a01f425abb877ba397378765', 'request_id': 'req-c1aa1551-89ce-4e6d-a861-a50d48c0ac8a'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:53.602391 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-c1aa1551-89ce-4e6d-a861-a50d48c0ac8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31b13cf1a01f425abb877ba397378765', 'request_id': 'req-c1aa1551-89ce-4e6d-a861-a50d48c0ac8a'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:53.612353 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-c1aa1551-89ce-4e6d-a861-a50d48c0ac8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31b13cf1a01f425abb877ba397378765', 'request_id': 'req-c1aa1551-89ce-4e6d-a861-a50d48c0ac8a'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:53.630093 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-c1aa1551-89ce-4e6d-a861-a50d48c0ac8a admin admin] The token cache is being invalidate because project 31b13cf1a01f425abb877ba397378765 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:43:53.630412 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-c1aa1551-89ce-4e6d-a861-a50d48c0ac8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:53.631459 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-c1aa1551-89ce-4e6d-a861-a50d48c0ac8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:53.633178 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 943/1891] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:43:53 2022] DELETE /identity/v3/projects/31b13cf1a01f425abb877ba397378765 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:53.637583 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f222cff-88eb-4d0c-a4eb-87ebf9e06764 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:53.637879 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f222cff-88eb-4d0c-a4eb-87ebf9e06764 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:53.638168 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f222cff-88eb-4d0c-a4eb-87ebf9e06764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:53.658381 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-1f222cff-88eb-4d0c-a4eb-87ebf9e06764 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:53.659617 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-1f222cff-88eb-4d0c-a4eb-87ebf9e06764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:53.661613 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bdd509-3d48-4a64-8494-71fdf5d2211d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:53.662332 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bdd509-3d48-4a64-8494-71fdf5d2211d admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:53.662626 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bdd509-3d48-4a64-8494-71fdf5d2211d admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:53.662911 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bdd509-3d48-4a64-8494-71fdf5d2211d admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:53.663622 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bdd509-3d48-4a64-8494-71fdf5d2211d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:53.665593 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bdd509-3d48-4a64-8494-71fdf5d2211d admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:53.677464 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 944/1892] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:43:53.686142 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3548bba6-aaff-4dfc-8078-cd0ed2abcaf5 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:53.686439 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3548bba6-aaff-4dfc-8078-cd0ed2abcaf5 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:53.686754 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3548bba6-aaff-4dfc-8078-cd0ed2abcaf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:53.698721 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 949/1893] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:43:53 2022] POST /identity/v3/projects => generated 387 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:43:53.702857 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bdd509-3d48-4a64-8494-71fdf5d2211d admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:53.705213 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-87bdd509-3d48-4a64-8494-71fdf5d2211d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:53.709345 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-3548bba6-aaff-4dfc-8078-cd0ed2abcaf5 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:53.710484 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-3548bba6-aaff-4dfc-8078-cd0ed2abcaf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:53.763993 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:53.774692 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:53.795047 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 945/1894] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:43:53.795953 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bdd509-3d48-4a64-8494-71fdf5d2211d admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:53.796071 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3548bba6-aaff-4dfc-8078-cd0ed2abcaf5 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:53.801213 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46629d9f-23ec-4948-9132-8780d789cedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:53.801927 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46629d9f-23ec-4948-9132-8780d789cedc admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:53.802284 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46629d9f-23ec-4948-9132-8780d789cedc admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:53.802629 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46629d9f-23ec-4948-9132-8780d789cedc admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:53.803397 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46629d9f-23ec-4948-9132-8780d789cedc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:53.805569 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46629d9f-23ec-4948-9132-8780d789cedc admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:53.805683 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3548bba6-aaff-4dfc-8078-cd0ed2abcaf5 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:53.810275 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897ea08-71da-419d-93cd-abc744da267d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:53.810941 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897ea08-71da-419d-93cd-abc744da267d admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:53.811233 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897ea08-71da-419d-93cd-abc744da267d admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:53.811518 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897ea08-71da-419d-93cd-abc744da267d admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:53.812828 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4897ea08-71da-419d-93cd-abc744da267d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:43:53.813167 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4897ea08-71da-419d-93cd-abc744da267d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:53.814866 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4897ea08-71da-419d-93cd-abc744da267d admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:53.831373 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 950/1895] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:43:53 2022] POST /identity/v3/users => generated 294 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:43:53.832655 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46629d9f-23ec-4948-9132-8780d789cedc admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:53.832972 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 946/1896] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:43:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:43:53.836816 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897ea08-71da-419d-93cd-abc744da267d admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:53.842931 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46629d9f-23ec-4948-9132-8780d789cedc admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:53.846468 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897ea08-71da-419d-93cd-abc744da267d admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:53.847830 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99faf15f-3aae-4531-ae14-380ebedfc534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:53.848493 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99faf15f-3aae-4531-ae14-380ebedfc534 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:53.848784 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99faf15f-3aae-4531-ae14-380ebedfc534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:53.849068 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99faf15f-3aae-4531-ae14-380ebedfc534 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:53.849867 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99faf15f-3aae-4531-ae14-380ebedfc534 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:53.851144 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeea34c-6064-45d4-8e9f-0007700027f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:53.851478 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99faf15f-3aae-4531-ae14-380ebedfc534 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:53.851781 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeea34c-6064-45d4-8e9f-0007700027f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:53.852066 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeea34c-6064-45d4-8e9f-0007700027f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:53.852345 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeea34c-6064-45d4-8e9f-0007700027f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:53.853031 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeea34c-6064-45d4-8e9f-0007700027f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:53.854950 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeea34c-6064-45d4-8e9f-0007700027f7 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:53.861697 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 951/1897] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:43:53.865555 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99faf15f-3aae-4531-ae14-380ebedfc534 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:53.875866 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99faf15f-3aae-4531-ae14-380ebedfc534 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:53.880585 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c782714-b5a6-4797-b34a-eea5e8ac8958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:53.881522 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c782714-b5a6-4797-b34a-eea5e8ac8958 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:53.881815 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c782714-b5a6-4797-b34a-eea5e8ac8958 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:53.882129 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c782714-b5a6-4797-b34a-eea5e8ac8958 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:53.882860 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c782714-b5a6-4797-b34a-eea5e8ac8958 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:53.886564 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 947/1898] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:43:53 2022] POST /identity/v3/projects => generated 407 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:43:53.886690 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c782714-b5a6-4797-b34a-eea5e8ac8958 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:53.887551 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeea34c-6064-45d4-8e9f-0007700027f7 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:53.903729 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 952/1899] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:43:53.908075 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c782714-b5a6-4797-b34a-eea5e8ac8958 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:53.918099 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c782714-b5a6-4797-b34a-eea5e8ac8958 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:53.956262 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeea34c-6064-45d4-8e9f-0007700027f7 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:53.965839 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e876cd-1d8a-4262-86fb-adbbe190ea22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:53.966530 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e876cd-1d8a-4262-86fb-adbbe190ea22 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:53.966819 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e876cd-1d8a-4262-86fb-adbbe190ea22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:53.967100 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e876cd-1d8a-4262-86fb-adbbe190ea22 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:53.967805 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e876cd-1d8a-4262-86fb-adbbe190ea22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:53.969811 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e876cd-1d8a-4262-86fb-adbbe190ea22 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:53.982989 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff1fb5-d6bf-4331-bc83-85126cc2d3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:53.984664 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff1fb5-d6bf-4331-bc83-85126cc2d3c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:53.985058 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff1fb5-d6bf-4331-bc83-85126cc2d3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:53.985322 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff1fb5-d6bf-4331-bc83-85126cc2d3c1 admin admin] PATH_INFO: `/v3/projects/bc31e8c2eb5b45f5bd5087c5a16a6d52/users/17d433c1d65b4d668132709029b80ac7/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:53.993723 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 948/1900] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:43:53 2022] POST /identity/v3/users => generated 304 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:43:53.995038 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e876cd-1d8a-4262-86fb-adbbe190ea22 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:54.002492 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ff1fb5-d6bf-4331-bc83-85126cc2d3c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc31e8c2eb5b45f5bd5087c5a16a6d52, user_id=17d433c1d65b4d668132709029b80ac7, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:54.005225 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ff1fb5-d6bf-4331-bc83-85126cc2d3c1 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:54.005453 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e876cd-1d8a-4262-86fb-adbbe190ea22 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:54.011057 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0163be-0a2e-4e56-b5f1-0d6b43519292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:54.011729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0163be-0a2e-4e56-b5f1-0d6b43519292 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:54.012015 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0163be-0a2e-4e56-b5f1-0d6b43519292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:54.012297 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0163be-0a2e-4e56-b5f1-0d6b43519292 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:54.013029 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0163be-0a2e-4e56-b5f1-0d6b43519292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:54.014674 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0163be-0a2e-4e56-b5f1-0d6b43519292 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:54.019064 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 953/1901] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:43:53 2022] PUT /identity/v3/projects/bc31e8c2eb5b45f5bd5087c5a16a6d52/users/17d433c1d65b4d668132709029b80ac7/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:43:54.026727 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 949/1902] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:43:54.028886 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0163be-0a2e-4e56-b5f1-0d6b43519292 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:54.029559 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f1e3b-743f-45e7-be75-efeb4aea3dc0 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:54.029857 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f1e3b-743f-45e7-be75-efeb4aea3dc0 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:54.030146 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f1e3b-743f-45e7-be75-efeb4aea3dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:54.038597 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0163be-0a2e-4e56-b5f1-0d6b43519292 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:54.056740 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-fb7f1e3b-743f-45e7-be75-efeb4aea3dc0 None None] MFA Rules not processed for user `17d433c1d65b4d668132709029b80ac7`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:54.057917 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-fb7f1e3b-743f-45e7-be75-efeb4aea3dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:54.094231 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd3aa03-2e32-4555-bddf-76e9039809a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:54.094905 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3aa03-2e32-4555-bddf-76e9039809a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:54.095200 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3aa03-2e32-4555-bddf-76e9039809a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:54.095483 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3aa03-2e32-4555-bddf-76e9039809a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:54.096232 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd3aa03-2e32-4555-bddf-76e9039809a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:54.097885 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd3aa03-2e32-4555-bddf-76e9039809a0 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:54.111761 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 950/1903] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:54 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:43:54.112936 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd3aa03-2e32-4555-bddf-76e9039809a0 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:54.122377 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd3aa03-2e32-4555-bddf-76e9039809a0 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:54.122817 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 954/1904] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:54.127645 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c814838-9364-4d0a-84f7-e4f2c79de985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:54.127783 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7f1e3b-743f-45e7-be75-efeb4aea3dc0 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:54.128531 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c814838-9364-4d0a-84f7-e4f2c79de985 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:54.128815 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c814838-9364-4d0a-84f7-e4f2c79de985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:54.129095 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c814838-9364-4d0a-84f7-e4f2c79de985 admin admin] PATH_INFO: `/v3/projects/3d47fa99bb924764bd6c15da44eb2a52/users/d32b6d161b8a4af48231d6afebdbd7e4/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:54.138475 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7f1e3b-743f-45e7-be75-efeb4aea3dc0 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:54.143093 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb898b4-3f6f-4a0c-b5ea-138f16487c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:54.143746 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb898b4-3f6f-4a0c-b5ea-138f16487c08 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:54.144029 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb898b4-3f6f-4a0c-b5ea-138f16487c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:54.144308 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb898b4-3f6f-4a0c-b5ea-138f16487c08 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:54.144716 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c814838-9364-4d0a-84f7-e4f2c79de985 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d47fa99bb924764bd6c15da44eb2a52, user_id=d32b6d161b8a4af48231d6afebdbd7e4, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:54.144999 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb898b4-3f6f-4a0c-b5ea-138f16487c08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:54.146972 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb898b4-3f6f-4a0c-b5ea-138f16487c08 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:54.148118 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c814838-9364-4d0a-84f7-e4f2c79de985 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:54.161000 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:54.162029 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 951/1905] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:43:54 2022] PUT /identity/v3/projects/3d47fa99bb924764bd6c15da44eb2a52/users/d32b6d161b8a4af48231d6afebdbd7e4/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:43:54.174673 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165534ec-c72a-4183-8454-1a918870df11 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:54.175004 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165534ec-c72a-4183-8454-1a918870df11 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:54.175247 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165534ec-c72a-4183-8454-1a918870df11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:54.180435 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 955/1906] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:43:54 2022] POST /identity/v3/projects => generated 389 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:43:54.181774 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb898b4-3f6f-4a0c-b5ea-138f16487c08 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:54.188025 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:54.211190 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-165534ec-c72a-4183-8454-1a918870df11 None None] MFA Rules not processed for user `d32b6d161b8a4af48231d6afebdbd7e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:54.212407 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-165534ec-c72a-4183-8454-1a918870df11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:54.261230 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb898b4-3f6f-4a0c-b5ea-138f16487c08 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:54.266127 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf09be3-28b0-4272-b7ab-f9c98ffecca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:54.266810 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf09be3-28b0-4272-b7ab-f9c98ffecca9 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:54.267096 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf09be3-28b0-4272-b7ab-f9c98ffecca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:54.267375 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf09be3-28b0-4272-b7ab-f9c98ffecca9 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:54.268084 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf09be3-28b0-4272-b7ab-f9c98ffecca9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:54.270122 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf09be3-28b0-4272-b7ab-f9c98ffecca9 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:54.289257 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 952/1907] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:54 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:43:54.292194 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 956/1908] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:43:54 2022] POST /identity/v3/users => generated 301 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:43:54.293424 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165534ec-c72a-4183-8454-1a918870df11 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:54.295430 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf09be3-28b0-4272-b7ab-f9c98ffecca9 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:54.305626 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165534ec-c72a-4183-8454-1a918870df11 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:54.309518 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf09be3-28b0-4272-b7ab-f9c98ffecca9 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:54.310507 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ccdd60-b8a6-4c02-b1b4-b14b54bbb49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:54.311214 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ccdd60-b8a6-4c02-b1b4-b14b54bbb49b admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:54.311503 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ccdd60-b8a6-4c02-b1b4-b14b54bbb49b admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:54.311909 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ccdd60-b8a6-4c02-b1b4-b14b54bbb49b admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:54.313247 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ccdd60-b8a6-4c02-b1b4-b14b54bbb49b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:54.315300 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ccdd60-b8a6-4c02-b1b4-b14b54bbb49b admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:54.316115 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955eb502-14a0-4215-b13f-f7fdce5548ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:54.316901 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955eb502-14a0-4215-b13f-f7fdce5548ef admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:54.317319 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955eb502-14a0-4215-b13f-f7fdce5548ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:54.317611 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955eb502-14a0-4215-b13f-f7fdce5548ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:54.318355 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955eb502-14a0-4215-b13f-f7fdce5548ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:54.319987 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955eb502-14a0-4215-b13f-f7fdce5548ef admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:54.331643 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 957/1909] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:43:54.336758 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955eb502-14a0-4215-b13f-f7fdce5548ef admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:54.348525 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955eb502-14a0-4215-b13f-f7fdce5548ef admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:54.353199 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 953/1910] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:43:54 2022] POST /identity/v3/projects => generated 407 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:43:54.354882 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931f393e-5975-442b-b4c1-fa4af6afe8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:54.355307 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ccdd60-b8a6-4c02-b1b4-b14b54bbb49b admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:54.355918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f393e-5975-442b-b4c1-fa4af6afe8d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:54.356204 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f393e-5975-442b-b4c1-fa4af6afe8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:54.356597 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f393e-5975-442b-b4c1-fa4af6afe8d9 admin admin] PATH_INFO: `/v3/projects/5f2bdd0a7e7a4e639de2de916c4b3455/users/6f6d60274a794d019116d7d9f96da341/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:54.375159 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931f393e-5975-442b-b4c1-fa4af6afe8d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f2bdd0a7e7a4e639de2de916c4b3455, user_id=6f6d60274a794d019116d7d9f96da341, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:54.377922 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931f393e-5975-442b-b4c1-fa4af6afe8d9 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:54.392397 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 958/1911] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:43:54 2022] PUT /identity/v3/projects/5f2bdd0a7e7a4e639de2de916c4b3455/users/6f6d60274a794d019116d7d9f96da341/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:43:54.396342 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931f393e-5975-442b-b4c1-fa4af6afe8d9 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:54.424033 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ccdd60-b8a6-4c02-b1b4-b14b54bbb49b admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:54.472617 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931f393e-5975-442b-b4c1-fa4af6afe8d9 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:54.479443 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eda4468-2fce-4bea-9d09-36e82e1d7462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:54.480285 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda4468-2fce-4bea-9d09-36e82e1d7462 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:54.480642 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda4468-2fce-4bea-9d09-36e82e1d7462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:54.481069 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda4468-2fce-4bea-9d09-36e82e1d7462 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:54.482168 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eda4468-2fce-4bea-9d09-36e82e1d7462 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:54.483959 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eda4468-2fce-4bea-9d09-36e82e1d7462 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:54.497380 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39408447-3b80-4dd3-8848-2fe25852e21a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:54.498384 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39408447-3b80-4dd3-8848-2fe25852e21a admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:54.498838 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39408447-3b80-4dd3-8848-2fe25852e21a admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:54.499123 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39408447-3b80-4dd3-8848-2fe25852e21a admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:54.499835 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39408447-3b80-4dd3-8848-2fe25852e21a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:54.502038 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39408447-3b80-4dd3-8848-2fe25852e21a admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:54.505503 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 959/1912] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:54 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:43:54.506707 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eda4468-2fce-4bea-9d09-36e82e1d7462 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:54.518613 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eda4468-2fce-4bea-9d09-36e82e1d7462 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:54.523417 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374ee5e-8802-44a2-bf94-4d768203d40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:54.526418 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 954/1913] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:43:54 2022] POST /identity/v3/users => generated 310 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:43:54.526868 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374ee5e-8802-44a2-bf94-4d768203d40b admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:54.526868 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374ee5e-8802-44a2-bf94-4d768203d40b admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:54.526868 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374ee5e-8802-44a2-bf94-4d768203d40b admin admin] PATH_INFO: `/v3/projects/5f2bdd0a7e7a4e639de2de916c4b3455/users/6f6d60274a794d019116d7d9f96da341/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:54.527117 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4374ee5e-8802-44a2-bf94-4d768203d40b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f2bdd0a7e7a4e639de2de916c4b3455, user_id=6f6d60274a794d019116d7d9f96da341, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:54.527415 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39408447-3b80-4dd3-8848-2fe25852e21a admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:54.529736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4374ee5e-8802-44a2-bf94-4d768203d40b admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:54.537551 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39408447-3b80-4dd3-8848-2fe25852e21a admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:54.541031 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 960/1914] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:43:54 2022] PUT /identity/v3/projects/5f2bdd0a7e7a4e639de2de916c4b3455/users/6f6d60274a794d019116d7d9f96da341/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:54.545993 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:54.552994 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927726b-e07d-4ca5-8df2-066ad7b42002 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:54.553330 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927726b-e07d-4ca5-8df2-066ad7b42002 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:54.553617 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927726b-e07d-4ca5-8df2-066ad7b42002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:54.581257 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-a927726b-e07d-4ca5-8df2-066ad7b42002 None None] MFA Rules not processed for user `6f6d60274a794d019116d7d9f96da341`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:54.582411 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-a927726b-e07d-4ca5-8df2-066ad7b42002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:54.599321 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ccf90a-76b1-4a1a-86e8-c9c70c9375ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:54.600002 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ccf90a-76b1-4a1a-86e8-c9c70c9375ea admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:54.600300 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ccf90a-76b1-4a1a-86e8-c9c70c9375ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:54.600586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ccf90a-76b1-4a1a-86e8-c9c70c9375ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:54.600735 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:54.601363 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ccf90a-76b1-4a1a-86e8-c9c70c9375ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:54.603154 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ccf90a-76b1-4a1a-86e8-c9c70c9375ea admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:54.614065 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 955/1915] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:54 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:43:54.618110 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ccf90a-76b1-4a1a-86e8-c9c70c9375ea admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:54.633428 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ccf90a-76b1-4a1a-86e8-c9c70c9375ea admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:54.639012 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83823de3-5fac-47f6-b602-788a1d97660d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:54.640145 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83823de3-5fac-47f6-b602-788a1d97660d admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:54.641031 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83823de3-5fac-47f6-b602-788a1d97660d admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:54.641320 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83823de3-5fac-47f6-b602-788a1d97660d admin admin] PATH_INFO: `/v3/projects/6a6cc6b3b6734db98c8aa814ec142eae/users/06766f4366524f929b2d44704ce2f455/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:54.656785 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83823de3-5fac-47f6-b602-788a1d97660d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a6cc6b3b6734db98c8aa814ec142eae, user_id=06766f4366524f929b2d44704ce2f455, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:54.658823 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 961/1916] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:54 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:43:54.659708 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83823de3-5fac-47f6-b602-788a1d97660d admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:54.670146 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927726b-e07d-4ca5-8df2-066ad7b42002 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:54.671875 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-a927726b-e07d-4ca5-8df2-066ad7b42002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:54.673273 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 956/1917] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:43:54 2022] PUT /identity/v3/projects/6a6cc6b3b6734db98c8aa814ec142eae/users/06766f4366524f929b2d44704ce2f455/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:43:54.677923 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83823de3-5fac-47f6-b602-788a1d97660d admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:54.738917 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927726b-e07d-4ca5-8df2-066ad7b42002 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:54.753098 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83823de3-5fac-47f6-b602-788a1d97660d admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:54.758820 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe22f89-5167-4626-b80b-1f83ec49f65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:54.759503 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe22f89-5167-4626-b80b-1f83ec49f65d admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:54.759791 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe22f89-5167-4626-b80b-1f83ec49f65d admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:54.760073 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe22f89-5167-4626-b80b-1f83ec49f65d admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:54.760802 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe22f89-5167-4626-b80b-1f83ec49f65d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:54.762630 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe22f89-5167-4626-b80b-1f83ec49f65d admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:54.778977 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 957/1918] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:54 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:43:54.779834 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe22f89-5167-4626-b80b-1f83ec49f65d admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:54.790831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe22f89-5167-4626-b80b-1f83ec49f65d admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:54.796075 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b6e37-ada8-40bf-8eb6-6675312aad86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:54.797595 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b6e37-ada8-40bf-8eb6-6675312aad86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:54.797881 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b6e37-ada8-40bf-8eb6-6675312aad86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:54.798161 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b6e37-ada8-40bf-8eb6-6675312aad86 admin admin] PATH_INFO: `/v3/projects/6a6cc6b3b6734db98c8aa814ec142eae/users/06766f4366524f929b2d44704ce2f455/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:54.799944 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d463a737-a514-4294-b73a-3b99c7d78d85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:54.800632 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d463a737-a514-4294-b73a-3b99c7d78d85 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:54.800745 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b6e37-ada8-40bf-8eb6-6675312aad86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a6cc6b3b6734db98c8aa814ec142eae, user_id=06766f4366524f929b2d44704ce2f455, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:54.800924 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d463a737-a514-4294-b73a-3b99c7d78d85 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:54.801713 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d463a737-a514-4294-b73a-3b99c7d78d85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:54.807382 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b6e37-ada8-40bf-8eb6-6675312aad86 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:54.820583 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 958/1919] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:43:54 2022] PUT /identity/v3/projects/6a6cc6b3b6734db98c8aa814ec142eae/users/06766f4366524f929b2d44704ce2f455/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:54.831442 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8fe74-ed44-4993-89d3-b7279dceff7e None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:54.831676 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8fe74-ed44-4993-89d3-b7279dceff7e None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:54.831956 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8fe74-ed44-4993-89d3-b7279dceff7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:54.871040 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-5fd8fe74-ed44-4993-89d3-b7279dceff7e None None] MFA Rules not processed for user `06766f4366524f929b2d44704ce2f455`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:54.873654 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-5fd8fe74-ed44-4993-89d3-b7279dceff7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:54.874056 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d463a737-a514-4294-b73a-3b99c7d78d85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:54.876162 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d463a737-a514-4294-b73a-3b99c7d78d85 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:54.887678 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 962/1920] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:54 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 16:43:54.949192 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:54.975571 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 959/1921] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:54 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:43:54.987296 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d463a737-a514-4294-b73a-3b99c7d78d85 service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:54.998228 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d463a737-a514-4294-b73a-3b99c7d78d85 service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:55.023264 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:55.046453 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd8fe74-ed44-4993-89d3-b7279dceff7e None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:55.062090 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab988d5b-29f6-4c33-8e17-49e2dbdfa1ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:55.063324 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab988d5b-29f6-4c33-8e17-49e2dbdfa1ed service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:55.063619 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab988d5b-29f6-4c33-8e17-49e2dbdfa1ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:55.063896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab988d5b-29f6-4c33-8e17-49e2dbdfa1ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:55.073640 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab988d5b-29f6-4c33-8e17-49e2dbdfa1ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:55.075123 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab988d5b-29f6-4c33-8e17-49e2dbdfa1ed service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:55.088896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 963/1922] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:54 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:43:55.125238 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd8fe74-ed44-4993-89d3-b7279dceff7e None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:55.131872 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1141c33-6b76-4263-b77b-859f3efe4b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:55.132712 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1141c33-6b76-4263-b77b-859f3efe4b0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:55.133001 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1141c33-6b76-4263-b77b-859f3efe4b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:55.133558 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1141c33-6b76-4263-b77b-859f3efe4b0d admin admin] PATH_INFO: `/v3/users/17d433c1d65b4d668132709029b80ac7` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:55.135030 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1141c33-6b76-4263-b77b-859f3efe4b0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=17d433c1d65b4d668132709029b80ac7)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:55.136918 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1141c33-6b76-4263-b77b-859f3efe4b0d admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:55.194228 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-c1141c33-6b76-4263-b77b-859f3efe4b0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17d433c1d65b4d668132709029b80ac7'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:55.195014 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-c1141c33-6b76-4263-b77b-859f3efe4b0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17d433c1d65b4d668132709029b80ac7'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:55.202881 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-c1141c33-6b76-4263-b77b-859f3efe4b0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17d433c1d65b4d668132709029b80ac7'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:55.216007 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-c1141c33-6b76-4263-b77b-859f3efe4b0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17d433c1d65b4d668132709029b80ac7'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:55.216918 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-c1141c33-6b76-4263-b77b-859f3efe4b0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17d433c1d65b4d668132709029b80ac7'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:55.230180 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 960/1923] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:43:55 2022] DELETE /identity/v3/users/17d433c1d65b4d668132709029b80ac7 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:43:55.235434 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b27766-0aa7-4a77-87db-b63fb14e5f7b None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:55.235728 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b27766-0aa7-4a77-87db-b63fb14e5f7b None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:55.236008 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b27766-0aa7-4a77-87db-b63fb14e5f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:55.253764 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1141c33-6b76-4263-b77b-859f3efe4b0d admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:55.254363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-a7b27766-0aa7-4a77-87db-b63fb14e5f7b None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:55.255262 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-c1141c33-6b76-4263-b77b-859f3efe4b0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:55.255543 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-a7b27766-0aa7-4a77-87db-b63fb14e5f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:55.315494 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1141c33-6b76-4263-b77b-859f3efe4b0d admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:55.316875 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 964/1924] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:55.319497 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2447bb5c-4442-4ec5-b4c6-d0f6568e21c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:55.320244 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447bb5c-4442-4ec5-b4c6-d0f6568e21c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:55.320546 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447bb5c-4442-4ec5-b4c6-d0f6568e21c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:55.320809 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447bb5c-4442-4ec5-b4c6-d0f6568e21c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:55.322701 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-2447bb5c-4442-4ec5-b4c6-d0f6568e21c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:55.326775 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b27766-0aa7-4a77-87db-b63fb14e5f7b None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:55.328307 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-a7b27766-0aa7-4a77-87db-b63fb14e5f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:55.354159 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:55.395345 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b27766-0aa7-4a77-87db-b63fb14e5f7b None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:55.395646 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2447bb5c-4442-4ec5-b4c6-d0f6568e21c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:55.397173 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2447bb5c-4442-4ec5-b4c6-d0f6568e21c5 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:55.399534 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acbd64f-2a89-446b-9547-c2b8d2669b2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:55.400316 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acbd64f-2a89-446b-9547-c2b8d2669b2c service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:55.400686 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acbd64f-2a89-446b-9547-c2b8d2669b2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:55.401689 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acbd64f-2a89-446b-9547-c2b8d2669b2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:55.407862 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 961/1925] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:55 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 16:43:55.416249 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acbd64f-2a89-446b-9547-c2b8d2669b2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:55.418153 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acbd64f-2a89-446b-9547-c2b8d2669b2c service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:55.429223 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 965/1926] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 16:43:55.810456 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2447bb5c-4442-4ec5-b4c6-d0f6568e21c5 service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:55.811944 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-2447bb5c-4442-4ec5-b4c6-d0f6568e21c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:55.828867 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2447bb5c-4442-4ec5-b4c6-d0f6568e21c5 service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:55.833738 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd2382-301b-46ab-be1e-ce6c839d1e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:55.834624 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd2382-301b-46ab-be1e-ce6c839d1e3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:55.834912 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd2382-301b-46ab-be1e-ce6c839d1e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:55.835194 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd2382-301b-46ab-be1e-ce6c839d1e3c admin admin] PATH_INFO: `/v3/projects/bc31e8c2eb5b45f5bd5087c5a16a6d52` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:55.836414 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fd2382-301b-46ab-be1e-ce6c839d1e3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc31e8c2eb5b45f5bd5087c5a16a6d52)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:55.838386 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fd2382-301b-46ab-be1e-ce6c839d1e3c admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:55.893760 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-73fd2382-301b-46ab-be1e-ce6c839d1e3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc31e8c2eb5b45f5bd5087c5a16a6d52', 'request_id': 'req-73fd2382-301b-46ab-be1e-ce6c839d1e3c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:55.900653 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-73fd2382-301b-46ab-be1e-ce6c839d1e3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc31e8c2eb5b45f5bd5087c5a16a6d52', 'request_id': 'req-73fd2382-301b-46ab-be1e-ce6c839d1e3c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:55.907481 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-73fd2382-301b-46ab-be1e-ce6c839d1e3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc31e8c2eb5b45f5bd5087c5a16a6d52', 'request_id': 'req-73fd2382-301b-46ab-be1e-ce6c839d1e3c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:55.920998 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-73fd2382-301b-46ab-be1e-ce6c839d1e3c admin admin] The token cache is being invalidate because project bc31e8c2eb5b45f5bd5087c5a16a6d52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:43:55.921332 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-73fd2382-301b-46ab-be1e-ce6c839d1e3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:55.922866 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-73fd2382-301b-46ab-be1e-ce6c839d1e3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:55.924620 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 962/1927] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:43:55 2022] DELETE /identity/v3/projects/bc31e8c2eb5b45f5bd5087c5a16a6d52 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:55.928622 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acbd64f-2a89-446b-9547-c2b8d2669b2c service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:55.940431 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acbd64f-2a89-446b-9547-c2b8d2669b2c service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:55.999099 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde76673-13d5-4d23-843e-1e559558da03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:56.000487 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde76673-13d5-4d23-843e-1e559558da03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:56.000771 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde76673-13d5-4d23-843e-1e559558da03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:56.001050 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde76673-13d5-4d23-843e-1e559558da03 admin admin] PATH_INFO: `/v3/users/6f6d60274a794d019116d7d9f96da341` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:56.002298 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde76673-13d5-4d23-843e-1e559558da03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f6d60274a794d019116d7d9f96da341)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:56.004352 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde76673-13d5-4d23-843e-1e559558da03 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:56.018398 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:56.066304 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-cde76673-13d5-4d23-843e-1e559558da03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f6d60274a794d019116d7d9f96da341'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:56.066989 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-cde76673-13d5-4d23-843e-1e559558da03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f6d60274a794d019116d7d9f96da341'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:56.067603 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-cde76673-13d5-4d23-843e-1e559558da03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f6d60274a794d019116d7d9f96da341'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:56.075436 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-cde76673-13d5-4d23-843e-1e559558da03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f6d60274a794d019116d7d9f96da341'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:56.088149 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-cde76673-13d5-4d23-843e-1e559558da03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f6d60274a794d019116d7d9f96da341'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:56.100231 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 966/1928] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:43:55 2022] DELETE /identity/v3/users/6f6d60274a794d019116d7d9f96da341 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:43:56.466395 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd2382-301b-46ab-be1e-ce6c839d1e3c admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:56.467874 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-73fd2382-301b-46ab-be1e-ce6c839d1e3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:56.482828 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd2382-301b-46ab-be1e-ce6c839d1e3c admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:56.540904 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143797de-82d1-4079-bd57-9dd643d1446e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:56.541809 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143797de-82d1-4079-bd57-9dd643d1446e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:56.542098 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143797de-82d1-4079-bd57-9dd643d1446e admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:56.542380 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143797de-82d1-4079-bd57-9dd643d1446e admin admin] PATH_INFO: `/v3/projects/5f2bdd0a7e7a4e639de2de916c4b3455` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:56.543598 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143797de-82d1-4079-bd57-9dd643d1446e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f2bdd0a7e7a4e639de2de916c4b3455)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:56.545563 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143797de-82d1-4079-bd57-9dd643d1446e admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:56.601577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-143797de-82d1-4079-bd57-9dd643d1446e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f2bdd0a7e7a4e639de2de916c4b3455', 'request_id': 'req-143797de-82d1-4079-bd57-9dd643d1446e'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:56.606996 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:56.610488 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-143797de-82d1-4079-bd57-9dd643d1446e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f2bdd0a7e7a4e639de2de916c4b3455', 'request_id': 'req-143797de-82d1-4079-bd57-9dd643d1446e'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:56.620162 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-143797de-82d1-4079-bd57-9dd643d1446e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f2bdd0a7e7a4e639de2de916c4b3455', 'request_id': 'req-143797de-82d1-4079-bd57-9dd643d1446e'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:56.637949 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-143797de-82d1-4079-bd57-9dd643d1446e admin admin] The token cache is being invalidate because project 5f2bdd0a7e7a4e639de2de916c4b3455 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:43:56.639154 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-143797de-82d1-4079-bd57-9dd643d1446e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:56.640863 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-143797de-82d1-4079-bd57-9dd643d1446e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:56.644422 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 963/1929] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:43:56 2022] DELETE /identity/v3/projects/5f2bdd0a7e7a4e639de2de916c4b3455 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:56.649483 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b5822-eb01-4a59-8b93-b5001ab16359 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:56.649782 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b5822-eb01-4a59-8b93-b5001ab16359 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:56.650063 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b5822-eb01-4a59-8b93-b5001ab16359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:56.673960 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-731b5822-eb01-4a59-8b93-b5001ab16359 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:56.675185 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-731b5822-eb01-4a59-8b93-b5001ab16359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:56.759976 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 967/1930] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:56.762229 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1b641-0d79-4d9f-b0b1-943c8ac076a0 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:56.762595 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1b641-0d79-4d9f-b0b1-943c8ac076a0 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:56.762889 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1b641-0d79-4d9f-b0b1-943c8ac076a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:56.783241 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-7ca1b641-0d79-4d9f-b0b1-943c8ac076a0 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:56.784536 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-7ca1b641-0d79-4d9f-b0b1-943c8ac076a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:56.802013 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 964/1931] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:56.806515 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731b5822-eb01-4a59-8b93-b5001ab16359 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:56.816135 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731b5822-eb01-4a59-8b93-b5001ab16359 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:56.820735 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7fae20-9e8a-4505-9f22-cc67f01f7b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:56.821415 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7fae20-9e8a-4505-9f22-cc67f01f7b40 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:56.821701 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7fae20-9e8a-4505-9f22-cc67f01f7b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:56.822003 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7fae20-9e8a-4505-9f22-cc67f01f7b40 admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:56.822722 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7fae20-9e8a-4505-9f22-cc67f01f7b40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:43:56.823036 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7fae20-9e8a-4505-9f22-cc67f01f7b40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:56.824636 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7fae20-9e8a-4505-9f22-cc67f01f7b40 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:56.836785 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 968/1932] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:43:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:43:56.840934 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1b641-0d79-4d9f-b0b1-943c8ac076a0 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:56.851030 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1b641-0d79-4d9f-b0b1-943c8ac076a0 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:56.855995 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b3e87-6f63-47f9-86c9-386dbd80bd93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:56.856729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b3e87-6f63-47f9-86c9-386dbd80bd93 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:56.857017 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b3e87-6f63-47f9-86c9-386dbd80bd93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:56.857343 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b3e87-6f63-47f9-86c9-386dbd80bd93 admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:56.858089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8b3e87-6f63-47f9-86c9-386dbd80bd93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:56.860656 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8b3e87-6f63-47f9-86c9-386dbd80bd93 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:56.886547 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 965/1933] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:43:56 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:43:56.890841 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7fae20-9e8a-4505-9f22-cc67f01f7b40 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:56.900983 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7fae20-9e8a-4505-9f22-cc67f01f7b40 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:56.959606 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9335c4f7-e64e-4b1b-813e-5143a16e9da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:56.960309 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335c4f7-e64e-4b1b-813e-5143a16e9da8 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:56.960554 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335c4f7-e64e-4b1b-813e-5143a16e9da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:56.960831 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335c4f7-e64e-4b1b-813e-5143a16e9da8 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:56.961567 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9335c4f7-e64e-4b1b-813e-5143a16e9da8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:56.963491 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9335c4f7-e64e-4b1b-813e-5143a16e9da8 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:56.983570 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 969/1934] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:43:56 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:43:56.988299 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b3e87-6f63-47f9-86c9-386dbd80bd93 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:56.998775 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b3e87-6f63-47f9-86c9-386dbd80bd93 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:57.004073 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc0d69d-a66d-4652-af0e-d4599fccd900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:57.004737 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc0d69d-a66d-4652-af0e-d4599fccd900 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:57.005026 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc0d69d-a66d-4652-af0e-d4599fccd900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:57.005340 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc0d69d-a66d-4652-af0e-d4599fccd900 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:57.006335 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc0d69d-a66d-4652-af0e-d4599fccd900 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:57.007943 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc0d69d-a66d-4652-af0e-d4599fccd900 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:57.020146 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 966/1935] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:43:57.024274 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9335c4f7-e64e-4b1b-813e-5143a16e9da8 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:57.030546 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:57.036716 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9335c4f7-e64e-4b1b-813e-5143a16e9da8 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:57.042491 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533f1de6-f2fe-4069-a3f8-7a205cb0d71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:57.043165 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f1de6-f2fe-4069-a3f8-7a205cb0d71e admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:57.043449 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f1de6-f2fe-4069-a3f8-7a205cb0d71e admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:57.043731 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f1de6-f2fe-4069-a3f8-7a205cb0d71e admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:57.044464 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533f1de6-f2fe-4069-a3f8-7a205cb0d71e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:57.046091 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533f1de6-f2fe-4069-a3f8-7a205cb0d71e admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:57.056619 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 970/1936] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:43:57.060920 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc0d69d-a66d-4652-af0e-d4599fccd900 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:57.070925 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc0d69d-a66d-4652-af0e-d4599fccd900 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:57.075653 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c4fc2-9c89-4332-bea0-43315658aae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:57.076548 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c4fc2-9c89-4332-bea0-43315658aae0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:57.076838 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c4fc2-9c89-4332-bea0-43315658aae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:57.077121 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c4fc2-9c89-4332-bea0-43315658aae0 admin admin] PATH_INFO: `/v3/projects/6c91c3f0f57f44ea969520be520a1e84/users/cbe28bd1467e42b5b5d2db0795df0987/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:57.092866 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0c4fc2-9c89-4332-bea0-43315658aae0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c91c3f0f57f44ea969520be520a1e84, user_id=cbe28bd1467e42b5b5d2db0795df0987, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:57.096328 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0c4fc2-9c89-4332-bea0-43315658aae0 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:57.110375 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 967/1937] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:43:57 2022] PUT /identity/v3/projects/6c91c3f0f57f44ea969520be520a1e84/users/cbe28bd1467e42b5b5d2db0795df0987/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:57.120489 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb53356-bedd-4efb-86ce-11034e96fc69 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:57.120791 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb53356-bedd-4efb-86ce-11034e96fc69 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:57.121075 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb53356-bedd-4efb-86ce-11034e96fc69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:57.149256 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-8cb53356-bedd-4efb-86ce-11034e96fc69 None None] MFA Rules not processed for user `cbe28bd1467e42b5b5d2db0795df0987`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:57.150432 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-8cb53356-bedd-4efb-86ce-11034e96fc69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:57.217197 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 971/1938] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:57.222323 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c4fc2-9c89-4332-bea0-43315658aae0 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:57.284685 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c4fc2-9c89-4332-bea0-43315658aae0 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:57.289384 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cf70b3-a7e7-433f-91ee-03fa9e84e5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:57.290046 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf70b3-a7e7-433f-91ee-03fa9e84e5d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:57.290334 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf70b3-a7e7-433f-91ee-03fa9e84e5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:57.290613 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf70b3-a7e7-433f-91ee-03fa9e84e5d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:57.291320 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cf70b3-a7e7-433f-91ee-03fa9e84e5d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:57.293256 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cf70b3-a7e7-433f-91ee-03fa9e84e5d8 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:57.318897 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 968/1939] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:43:57 2022] POST /identity/v3/projects => generated 381 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:43:57.323004 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb53356-bedd-4efb-86ce-11034e96fc69 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:57.333676 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb53356-bedd-4efb-86ce-11034e96fc69 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:57.388855 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95047357-1827-4751-b0ff-3ad7a7c5de27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:57.389556 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95047357-1827-4751-b0ff-3ad7a7c5de27 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:57.389842 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95047357-1827-4751-b0ff-3ad7a7c5de27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:57.390125 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95047357-1827-4751-b0ff-3ad7a7c5de27 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:57.390859 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95047357-1827-4751-b0ff-3ad7a7c5de27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:57.392795 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95047357-1827-4751-b0ff-3ad7a7c5de27 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:57.412793 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 972/1940] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:43:57 2022] POST /identity/v3/users => generated 297 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:43:57.416260 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cf70b3-a7e7-433f-91ee-03fa9e84e5d8 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:57.427200 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cf70b3-a7e7-433f-91ee-03fa9e84e5d8 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:57.431825 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c08e7-ceec-4414-be99-c8d1803cf6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:57.432490 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c08e7-ceec-4414-be99-c8d1803cf6f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:57.432795 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c08e7-ceec-4414-be99-c8d1803cf6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:57.433058 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c08e7-ceec-4414-be99-c8d1803cf6f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:57.433853 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6c08e7-ceec-4414-be99-c8d1803cf6f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:57.435451 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6c08e7-ceec-4414-be99-c8d1803cf6f8 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:57.448214 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 969/1941] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:43:57.452199 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95047357-1827-4751-b0ff-3ad7a7c5de27 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:57.462489 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95047357-1827-4751-b0ff-3ad7a7c5de27 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:57.467300 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d977e6-c3b1-4d07-96da-56549f0f88ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:57.468169 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d977e6-c3b1-4d07-96da-56549f0f88ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:57.469014 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d977e6-c3b1-4d07-96da-56549f0f88ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:57.469328 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d977e6-c3b1-4d07-96da-56549f0f88ec admin admin] PATH_INFO: `/v3/projects/04028d805922484d9dc270001787ebea/users/d6622d38cdf8463a9a6445cd515fca05/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:57.486849 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d977e6-c3b1-4d07-96da-56549f0f88ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=04028d805922484d9dc270001787ebea, user_id=d6622d38cdf8463a9a6445cd515fca05, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:57.489596 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d977e6-c3b1-4d07-96da-56549f0f88ec admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:57.503373 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 973/1942] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:43:57 2022] PUT /identity/v3/projects/04028d805922484d9dc270001787ebea/users/d6622d38cdf8463a9a6445cd515fca05/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:57.507833 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c08e7-ceec-4414-be99-c8d1803cf6f8 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:57.518231 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c08e7-ceec-4414-be99-c8d1803cf6f8 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:57.570346 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:57.582674 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d7de5-d8ed-4040-8c51-70ec39064679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:57.583542 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d7de5-d8ed-4040-8c51-70ec39064679 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:57.583871 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d7de5-d8ed-4040-8c51-70ec39064679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:57.584131 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d7de5-d8ed-4040-8c51-70ec39064679 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:57.585039 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03d7de5-d8ed-4040-8c51-70ec39064679 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:57.586809 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03d7de5-d8ed-4040-8c51-70ec39064679 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:57.599026 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 970/1943] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:43:57 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:43:57.603298 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d977e6-c3b1-4d07-96da-56549f0f88ec admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:57.614852 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d977e6-c3b1-4d07-96da-56549f0f88ec admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:57.619515 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b86ea-97f2-4f7c-a02e-a9cb3b24f287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:57.620496 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b86ea-97f2-4f7c-a02e-a9cb3b24f287 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:57.620824 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b86ea-97f2-4f7c-a02e-a9cb3b24f287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:57.621105 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b86ea-97f2-4f7c-a02e-a9cb3b24f287 admin admin] PATH_INFO: `/v3/projects/04028d805922484d9dc270001787ebea/users/d6622d38cdf8463a9a6445cd515fca05/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:57.623307 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b86ea-97f2-4f7c-a02e-a9cb3b24f287 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04028d805922484d9dc270001787ebea, user_id=d6622d38cdf8463a9a6445cd515fca05, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:57.626745 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b86ea-97f2-4f7c-a02e-a9cb3b24f287 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:57.639650 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 974/1944] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:43:57 2022] PUT /identity/v3/projects/04028d805922484d9dc270001787ebea/users/d6622d38cdf8463a9a6445cd515fca05/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:57.651438 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c778c9-18bd-4b4b-abdc-072ac7cba71c None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:57.651438 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c778c9-18bd-4b4b-abdc-072ac7cba71c None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:57.651583 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c778c9-18bd-4b4b-abdc-072ac7cba71c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:57.682571 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-76c778c9-18bd-4b4b-abdc-072ac7cba71c None None] MFA Rules not processed for user `d6622d38cdf8463a9a6445cd515fca05`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:57.683789 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-76c778c9-18bd-4b4b-abdc-072ac7cba71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:57.771005 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 971/1945] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:57 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:57.782325 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b86ea-97f2-4f7c-a02e-a9cb3b24f287 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:57.784044 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-688b86ea-97f2-4f7c-a02e-a9cb3b24f287 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:57.828734 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:57.852379 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b86ea-97f2-4f7c-a02e-a9cb3b24f287 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:57.855998 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75780ac-38a7-4973-912e-1cdc63beb7d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:57.856712 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75780ac-38a7-4973-912e-1cdc63beb7d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:57.857003 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75780ac-38a7-4973-912e-1cdc63beb7d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:57.857327 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75780ac-38a7-4973-912e-1cdc63beb7d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:57.872387 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75780ac-38a7-4973-912e-1cdc63beb7d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:57.874234 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75780ac-38a7-4973-912e-1cdc63beb7d2 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:57.885648 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 975/1946] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:57 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:43:57.998891 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c778c9-18bd-4b4b-abdc-072ac7cba71c None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:58.013386 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c778c9-18bd-4b4b-abdc-072ac7cba71c None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:58.016766 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb69bfa-c79e-467b-b452-821384fd2a02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:58.017588 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb69bfa-c79e-467b-b452-821384fd2a02 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:58.017876 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb69bfa-c79e-467b-b452-821384fd2a02 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:58.018208 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb69bfa-c79e-467b-b452-821384fd2a02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:58.020217 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-bcb69bfa-c79e-467b-b452-821384fd2a02 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:58.100578 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb69bfa-c79e-467b-b452-821384fd2a02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:58.102155 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb69bfa-c79e-467b-b452-821384fd2a02 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:58.113279 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 972/1947] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:57 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:43:58.588475 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75780ac-38a7-4973-912e-1cdc63beb7d2 service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:58.652795 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75780ac-38a7-4973-912e-1cdc63beb7d2 service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:58.657375 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aaf4d6-6e3d-4786-8358-84668ac7b682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:58.658250 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aaf4d6-6e3d-4786-8358-84668ac7b682 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:58.658559 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aaf4d6-6e3d-4786-8358-84668ac7b682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:58.658835 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aaf4d6-6e3d-4786-8358-84668ac7b682 admin admin] PATH_INFO: `/v3/users/cbe28bd1467e42b5b5d2db0795df0987` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:58.660126 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aaf4d6-6e3d-4786-8358-84668ac7b682 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbe28bd1467e42b5b5d2db0795df0987)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:58.662127 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aaf4d6-6e3d-4786-8358-84668ac7b682 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:58.711778 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-76aaf4d6-6e3d-4786-8358-84668ac7b682 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbe28bd1467e42b5b5d2db0795df0987'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:58.712648 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-76aaf4d6-6e3d-4786-8358-84668ac7b682 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbe28bd1467e42b5b5d2db0795df0987'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:58.713465 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-76aaf4d6-6e3d-4786-8358-84668ac7b682 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbe28bd1467e42b5b5d2db0795df0987'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:58.722913 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-76aaf4d6-6e3d-4786-8358-84668ac7b682 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbe28bd1467e42b5b5d2db0795df0987'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:58.736542 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-76aaf4d6-6e3d-4786-8358-84668ac7b682 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbe28bd1467e42b5b5d2db0795df0987'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:58.758312 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 976/1948] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:43:58 2022] DELETE /identity/v3/users/cbe28bd1467e42b5b5d2db0795df0987 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:58.762494 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ce8ea-ea2a-4b2a-a803-7f268c39d9e5 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:58.762788 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ce8ea-ea2a-4b2a-a803-7f268c39d9e5 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:58.763098 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ce8ea-ea2a-4b2a-a803-7f268c39d9e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:58.784222 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-ee2ce8ea-ea2a-4b2a-a803-7f268c39d9e5 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:58.786164 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ee2ce8ea-ea2a-4b2a-a803-7f268c39d9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:58.832971 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aaf4d6-6e3d-4786-8358-84668ac7b682 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:58.834698 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-76aaf4d6-6e3d-4786-8358-84668ac7b682 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:58.863411 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 973/1949] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:43:58.873539 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2ce8ea-ea2a-4b2a-a803-7f268c39d9e5 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:58.874964 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ee2ce8ea-ea2a-4b2a-a803-7f268c39d9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:58.882651 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:58.911337 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:58.915527 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aaf4d6-6e3d-4786-8358-84668ac7b682 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:58.928034 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81f5a6e-33ca-4fab-aa49-f191144493c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:58.928034 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f5a6e-33ca-4fab-aa49-f191144493c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:58.928034 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f5a6e-33ca-4fab-aa49-f191144493c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:58.928034 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f5a6e-33ca-4fab-aa49-f191144493c7 admin admin] PATH_INFO: `/v3/users/45ac490771d745a98754be293cceb749` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:58.931105 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81f5a6e-33ca-4fab-aa49-f191144493c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45ac490771d745a98754be293cceb749)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:58.933525 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81f5a6e-33ca-4fab-aa49-f191144493c7 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:58.940986 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2ce8ea-ea2a-4b2a-a803-7f268c39d9e5 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:58.944612 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdd61fe-d42f-48b1-80bd-0982eb257ae8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:58.945307 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd61fe-d42f-48b1-80bd-0982eb257ae8 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:58.945600 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd61fe-d42f-48b1-80bd-0982eb257ae8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:58.945876 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd61fe-d42f-48b1-80bd-0982eb257ae8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:58.955621 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdd61fe-d42f-48b1-80bd-0982eb257ae8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:58.957071 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdd61fe-d42f-48b1-80bd-0982eb257ae8 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:58.975033 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 974/1950] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 16:43:58.988353 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a81f5a6e-33ca-4fab-aa49-f191144493c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45ac490771d745a98754be293cceb749'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:58.989111 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a81f5a6e-33ca-4fab-aa49-f191144493c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45ac490771d745a98754be293cceb749'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:58.989837 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a81f5a6e-33ca-4fab-aa49-f191144493c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45ac490771d745a98754be293cceb749'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:58.997735 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a81f5a6e-33ca-4fab-aa49-f191144493c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45ac490771d745a98754be293cceb749'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:59.011857 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a81f5a6e-33ca-4fab-aa49-f191144493c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45ac490771d745a98754be293cceb749'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:59.033590 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 977/1951] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:43:58 2022] DELETE /identity/v3/users/45ac490771d745a98754be293cceb749 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:43:59.040818 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c981f-7b95-4e8b-ab4b-dcb24e1a0bd0 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:59.041804 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c981f-7b95-4e8b-ab4b-dcb24e1a0bd0 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:59.042275 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c981f-7b95-4e8b-ab4b-dcb24e1a0bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:59.074905 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-ba5c981f-7b95-4e8b-ab4b-dcb24e1a0bd0 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:43:59.076310 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ba5c981f-7b95-4e8b-ab4b-dcb24e1a0bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:59.162038 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 975/1952] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:43:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:43:59.175303 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81f5a6e-33ca-4fab-aa49-f191144493c7 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:59.176934 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-a81f5a6e-33ca-4fab-aa49-f191144493c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:59.258111 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81f5a6e-33ca-4fab-aa49-f191144493c7 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:59.261746 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f51cde0-7ebb-4923-8721-25dd919d3547 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:59.262414 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51cde0-7ebb-4923-8721-25dd919d3547 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:59.262700 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51cde0-7ebb-4923-8721-25dd919d3547 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:59.262991 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51cde0-7ebb-4923-8721-25dd919d3547 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:59.273050 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f51cde0-7ebb-4923-8721-25dd919d3547 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:59.274561 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f51cde0-7ebb-4923-8721-25dd919d3547 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:59.287895 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 978/1953] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:43:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:43:59.315194 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c981f-7b95-4e8b-ab4b-dcb24e1a0bd0 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:59.316690 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ba5c981f-7b95-4e8b-ab4b-dcb24e1a0bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:59.331179 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c981f-7b95-4e8b-ab4b-dcb24e1a0bd0 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:59.341196 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93d0d03-9df3-47f2-960f-ab98d7bb65ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:59.342376 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d0d03-9df3-47f2-960f-ab98d7bb65ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:59.342739 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d0d03-9df3-47f2-960f-ab98d7bb65ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:59.344229 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d0d03-9df3-47f2-960f-ab98d7bb65ef admin admin] PATH_INFO: `/v3/projects/6c91c3f0f57f44ea969520be520a1e84` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:59.346239 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93d0d03-9df3-47f2-960f-ab98d7bb65ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c91c3f0f57f44ea969520be520a1e84)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:59.348348 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93d0d03-9df3-47f2-960f-ab98d7bb65ef admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:59.403956 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f93d0d03-9df3-47f2-960f-ab98d7bb65ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c91c3f0f57f44ea969520be520a1e84', 'request_id': 'req-f93d0d03-9df3-47f2-960f-ab98d7bb65ef'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:59.410867 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f93d0d03-9df3-47f2-960f-ab98d7bb65ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c91c3f0f57f44ea969520be520a1e84', 'request_id': 'req-f93d0d03-9df3-47f2-960f-ab98d7bb65ef'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:59.423346 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f93d0d03-9df3-47f2-960f-ab98d7bb65ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c91c3f0f57f44ea969520be520a1e84', 'request_id': 'req-f93d0d03-9df3-47f2-960f-ab98d7bb65ef'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:59.437925 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f93d0d03-9df3-47f2-960f-ab98d7bb65ef admin admin] The token cache is being invalidate because project 6c91c3f0f57f44ea969520be520a1e84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:43:59.438165 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f93d0d03-9df3-47f2-960f-ab98d7bb65ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:59.438894 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f93d0d03-9df3-47f2-960f-ab98d7bb65ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:59.440603 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 976/1954] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:43:59 2022] DELETE /identity/v3/projects/6c91c3f0f57f44ea969520be520a1e84 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:59.448613 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f51cde0-7ebb-4923-8721-25dd919d3547 service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:59.459876 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f51cde0-7ebb-4923-8721-25dd919d3547 service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:59.535349 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0dd33-6ed4-4a2c-85c4-152f281ef666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:59.536241 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0dd33-6ed4-4a2c-85c4-152f281ef666 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:59.536524 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0dd33-6ed4-4a2c-85c4-152f281ef666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:59.536812 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0dd33-6ed4-4a2c-85c4-152f281ef666 admin admin] PATH_INFO: `/v3/users/d6622d38cdf8463a9a6445cd515fca05` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:59.538416 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd0dd33-6ed4-4a2c-85c4-152f281ef666 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6622d38cdf8463a9a6445cd515fca05)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:59.540592 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd0dd33-6ed4-4a2c-85c4-152f281ef666 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:59.569647 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:59.614007 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-8fd0dd33-6ed4-4a2c-85c4-152f281ef666 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6622d38cdf8463a9a6445cd515fca05'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:59.615877 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93d0d03-9df3-47f2-960f-ab98d7bb65ef admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:59.617735 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-f93d0d03-9df3-47f2-960f-ab98d7bb65ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:59.626233 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-8fd0dd33-6ed4-4a2c-85c4-152f281ef666 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6622d38cdf8463a9a6445cd515fca05'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:59.628538 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-8fd0dd33-6ed4-4a2c-85c4-152f281ef666 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6622d38cdf8463a9a6445cd515fca05'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:59.636502 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93d0d03-9df3-47f2-960f-ab98d7bb65ef admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:59.641651 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-8fd0dd33-6ed4-4a2c-85c4-152f281ef666 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6622d38cdf8463a9a6445cd515fca05'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:59.644785 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6538dc-0fb7-40fa-ae02-0c6420ae1ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:59.645439 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6538dc-0fb7-40fa-ae02-0c6420ae1ee5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:59.645766 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6538dc-0fb7-40fa-ae02-0c6420ae1ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:59.646178 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6538dc-0fb7-40fa-ae02-0c6420ae1ee5 admin admin] PATH_INFO: `/v3/projects/a93efcc3f25a496a9a093d6835ce3cb3` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:59.647614 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6538dc-0fb7-40fa-ae02-0c6420ae1ee5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a93efcc3f25a496a9a093d6835ce3cb3)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:59.649675 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6538dc-0fb7-40fa-ae02-0c6420ae1ee5 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:43:59.663918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-8fd0dd33-6ed4-4a2c-85c4-152f281ef666 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6622d38cdf8463a9a6445cd515fca05'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:59.677511 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 979/1955] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:43:59 2022] DELETE /identity/v3/users/d6622d38cdf8463a9a6445cd515fca05 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:59.690066 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:43:59.713923 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-bf6538dc-0fb7-40fa-ae02-0c6420ae1ee5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a93efcc3f25a496a9a093d6835ce3cb3', 'request_id': 'req-bf6538dc-0fb7-40fa-ae02-0c6420ae1ee5'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:59.722690 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-bf6538dc-0fb7-40fa-ae02-0c6420ae1ee5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a93efcc3f25a496a9a093d6835ce3cb3', 'request_id': 'req-bf6538dc-0fb7-40fa-ae02-0c6420ae1ee5'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:59.736206 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-bf6538dc-0fb7-40fa-ae02-0c6420ae1ee5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a93efcc3f25a496a9a093d6835ce3cb3', 'request_id': 'req-bf6538dc-0fb7-40fa-ae02-0c6420ae1ee5'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:59.750663 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-bf6538dc-0fb7-40fa-ae02-0c6420ae1ee5 admin admin] The token cache is being invalidate because project a93efcc3f25a496a9a093d6835ce3cb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:43:59.751053 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-bf6538dc-0fb7-40fa-ae02-0c6420ae1ee5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:59.752115 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-bf6538dc-0fb7-40fa-ae02-0c6420ae1ee5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:43:59.754210 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 977/1956] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:43:59 2022] DELETE /identity/v3/projects/a93efcc3f25a496a9a093d6835ce3cb3 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:43:59.758696 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0dd33-6ed4-4a2c-85c4-152f281ef666 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:43:59.760378 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-8fd0dd33-6ed4-4a2c-85c4-152f281ef666 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:43:59.850191 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0dd33-6ed4-4a2c-85c4-152f281ef666 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:43:59.932086 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493db97a-272f-41cf-a113-def44fef1c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:43:59.933169 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493db97a-272f-41cf-a113-def44fef1c12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:43:59.933566 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493db97a-272f-41cf-a113-def44fef1c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:43:59.933953 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493db97a-272f-41cf-a113-def44fef1c12 admin admin] PATH_INFO: `/v3/users/0ed942914a324076aaad268808717508` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:43:59.935453 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493db97a-272f-41cf-a113-def44fef1c12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ed942914a324076aaad268808717508)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:43:59.938952 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493db97a-272f-41cf-a113-def44fef1c12 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:00.000308 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-493db97a-272f-41cf-a113-def44fef1c12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ed942914a324076aaad268808717508'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:00.001061 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-493db97a-272f-41cf-a113-def44fef1c12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ed942914a324076aaad268808717508'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:00.001737 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-493db97a-272f-41cf-a113-def44fef1c12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ed942914a324076aaad268808717508'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:00.004737 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:00.031706 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-493db97a-272f-41cf-a113-def44fef1c12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ed942914a324076aaad268808717508'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:00.047775 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-493db97a-272f-41cf-a113-def44fef1c12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ed942914a324076aaad268808717508'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:00.061971 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 980/1957] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:43:59 2022] DELETE /identity/v3/users/0ed942914a324076aaad268808717508 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:00.111453 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6538dc-0fb7-40fa-ae02-0c6420ae1ee5 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:00.113823 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-bf6538dc-0fb7-40fa-ae02-0c6420ae1ee5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:00.143494 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6538dc-0fb7-40fa-ae02-0c6420ae1ee5 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:00.215816 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee34255-59f2-4bf4-b239-caef9da1882b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:00.216751 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee34255-59f2-4bf4-b239-caef9da1882b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:00.217418 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee34255-59f2-4bf4-b239-caef9da1882b admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:00.217418 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee34255-59f2-4bf4-b239-caef9da1882b admin admin] PATH_INFO: `/v3/projects/04028d805922484d9dc270001787ebea` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:00.218655 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee34255-59f2-4bf4-b239-caef9da1882b admin admin] RBAC: Authorizing `identity:delete_project(project_id=04028d805922484d9dc270001787ebea)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:00.220749 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee34255-59f2-4bf4-b239-caef9da1882b admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:00.299492 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-cee34255-59f2-4bf4-b239-caef9da1882b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04028d805922484d9dc270001787ebea', 'request_id': 'req-cee34255-59f2-4bf4-b239-caef9da1882b'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:00.306614 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-cee34255-59f2-4bf4-b239-caef9da1882b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04028d805922484d9dc270001787ebea', 'request_id': 'req-cee34255-59f2-4bf4-b239-caef9da1882b'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:00.315516 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-cee34255-59f2-4bf4-b239-caef9da1882b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04028d805922484d9dc270001787ebea', 'request_id': 'req-cee34255-59f2-4bf4-b239-caef9da1882b'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:00.333400 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-cee34255-59f2-4bf4-b239-caef9da1882b admin admin] The token cache is being invalidate because project 04028d805922484d9dc270001787ebea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:44:00.333788 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-cee34255-59f2-4bf4-b239-caef9da1882b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:00.334653 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-cee34255-59f2-4bf4-b239-caef9da1882b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:00.337111 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 978/1958] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:44:00 2022] DELETE /identity/v3/projects/04028d805922484d9dc270001787ebea => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:00.342294 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cb11dc-5118-4178-a28a-4c8d258381c7 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:00.342294 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cb11dc-5118-4178-a28a-4c8d258381c7 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:00.344180 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cb11dc-5118-4178-a28a-4c8d258381c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:00.363722 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-a0cb11dc-5118-4178-a28a-4c8d258381c7 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:00.365065 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-a0cb11dc-5118-4178-a28a-4c8d258381c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:00.398233 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee34255-59f2-4bf4-b239-caef9da1882b admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:00.400367 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-cee34255-59f2-4bf4-b239-caef9da1882b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:00.476092 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 981/1959] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:44:00.482471 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:00.484777 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c936c2e-d03e-4bb4-ac2c-fdb1a1a02aec None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:00.485068 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c936c2e-d03e-4bb4-ac2c-fdb1a1a02aec None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:00.485404 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c936c2e-d03e-4bb4-ac2c-fdb1a1a02aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:00.502128 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee34255-59f2-4bf4-b239-caef9da1882b admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:00.507542 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf50894f-de7a-4319-a87f-cc38ead7469e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:00.507698 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-9c936c2e-d03e-4bb4-ac2c-fdb1a1a02aec None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:00.508544 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf50894f-de7a-4319-a87f-cc38ead7469e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:00.508782 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-9c936c2e-d03e-4bb4-ac2c-fdb1a1a02aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:00.509019 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf50894f-de7a-4319-a87f-cc38ead7469e admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:00.509434 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf50894f-de7a-4319-a87f-cc38ead7469e admin admin] PATH_INFO: `/v3/projects/8d15182c742047e19af57f2f4d466192` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:00.510760 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf50894f-de7a-4319-a87f-cc38ead7469e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d15182c742047e19af57f2f4d466192)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:00.513164 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf50894f-de7a-4319-a87f-cc38ead7469e admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:00.516940 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 982/1960] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:44:00.521774 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c936c2e-d03e-4bb4-ac2c-fdb1a1a02aec None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:00.531911 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c936c2e-d03e-4bb4-ac2c-fdb1a1a02aec None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:00.537110 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e942daca-3b8f-4808-a522-677ea425220d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:00.537809 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942daca-3b8f-4808-a522-677ea425220d admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:00.538102 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942daca-3b8f-4808-a522-677ea425220d admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:00.538387 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942daca-3b8f-4808-a522-677ea425220d admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:00.539165 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e942daca-3b8f-4808-a522-677ea425220d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:44:00.539481 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e942daca-3b8f-4808-a522-677ea425220d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:00.541134 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e942daca-3b8f-4808-a522-677ea425220d admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:00.556096 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 983/1961] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:44:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:44:00.560253 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e942daca-3b8f-4808-a522-677ea425220d admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:00.570708 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e942daca-3b8f-4808-a522-677ea425220d admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:00.573666 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-bf50894f-de7a-4319-a87f-cc38ead7469e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d15182c742047e19af57f2f4d466192', 'request_id': 'req-bf50894f-de7a-4319-a87f-cc38ead7469e'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:00.581001 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-bf50894f-de7a-4319-a87f-cc38ead7469e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d15182c742047e19af57f2f4d466192', 'request_id': 'req-bf50894f-de7a-4319-a87f-cc38ead7469e'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:00.587908 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-bf50894f-de7a-4319-a87f-cc38ead7469e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d15182c742047e19af57f2f4d466192', 'request_id': 'req-bf50894f-de7a-4319-a87f-cc38ead7469e'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:00.605051 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-bf50894f-de7a-4319-a87f-cc38ead7469e admin admin] The token cache is being invalidate because project 8d15182c742047e19af57f2f4d466192 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:44:00.605389 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-bf50894f-de7a-4319-a87f-cc38ead7469e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:00.606112 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-bf50894f-de7a-4319-a87f-cc38ead7469e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:00.615146 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 979/1962] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:44:00 2022] DELETE /identity/v3/projects/8d15182c742047e19af57f2f4d466192 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:44:00.617543 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c5a17-5cbb-4f30-a274-1e0c1a9ffa6d None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:00.617834 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c5a17-5cbb-4f30-a274-1e0c1a9ffa6d None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:00.618125 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c5a17-5cbb-4f30-a274-1e0c1a9ffa6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:00.640644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-e52c5a17-5cbb-4f30-a274-1e0c1a9ffa6d None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:00.641402 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa50b7-3856-44bc-869e-bf9b37fcd9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:00.642594 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa50b7-3856-44bc-869e-bf9b37fcd9b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:00.642759 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-e52c5a17-5cbb-4f30-a274-1e0c1a9ffa6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:00.643178 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa50b7-3856-44bc-869e-bf9b37fcd9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:00.643805 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa50b7-3856-44bc-869e-bf9b37fcd9b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:00.644950 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aa50b7-3856-44bc-869e-bf9b37fcd9b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:00.647888 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aa50b7-3856-44bc-869e-bf9b37fcd9b3 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:00.652648 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 980/1963] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:44:00.667172 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69211ed-9bdf-43c4-9f51-f6a3f1fe7508 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:00.667172 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69211ed-9bdf-43c4-9f51-f6a3f1fe7508 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:00.667172 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69211ed-9bdf-43c4-9f51-f6a3f1fe7508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:00.687047 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-b69211ed-9bdf-43c4-9f51-f6a3f1fe7508 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:00.687717 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 984/1964] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:44:00 2022] POST /identity/v3/projects => generated 375 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:44:00.688537 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-b69211ed-9bdf-43c4-9f51-f6a3f1fe7508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:00.691905 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa50b7-3856-44bc-869e-bf9b37fcd9b3 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:00.694230 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b6aa50b7-3856-44bc-869e-bf9b37fcd9b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:00.739874 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:00.763009 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 981/1965] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:44:00.767954 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69211ed-9bdf-43c4-9f51-f6a3f1fe7508 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:00.777564 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69211ed-9bdf-43c4-9f51-f6a3f1fe7508 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:00.783372 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c20ce1-4307-493f-b14a-f2f809ace3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:00.784067 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c20ce1-4307-493f-b14a-f2f809ace3ba admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:00.784481 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c20ce1-4307-493f-b14a-f2f809ace3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:00.784783 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c20ce1-4307-493f-b14a-f2f809ace3ba admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:00.785561 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c20ce1-4307-493f-b14a-f2f809ace3ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:44:00.785875 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c20ce1-4307-493f-b14a-f2f809ace3ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:00.787507 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c20ce1-4307-493f-b14a-f2f809ace3ba admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:00.789463 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa50b7-3856-44bc-869e-bf9b37fcd9b3 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:00.796389 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f67931b-fa41-4443-84ce-07a8f8f0cbd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:00.797310 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67931b-fa41-4443-84ce-07a8f8f0cbd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:00.797745 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67931b-fa41-4443-84ce-07a8f8f0cbd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:00.798171 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67931b-fa41-4443-84ce-07a8f8f0cbd5 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:00.799266 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f67931b-fa41-4443-84ce-07a8f8f0cbd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:00.802400 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 982/1966] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:44:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:44:00.802564 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f67931b-fa41-4443-84ce-07a8f8f0cbd5 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:00.805694 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c20ce1-4307-493f-b14a-f2f809ace3ba admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:00.816828 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c20ce1-4307-493f-b14a-f2f809ace3ba admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:00.821796 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3796a0-7386-4ad4-ad90-76b31b9c6aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:00.822511 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3796a0-7386-4ad4-ad90-76b31b9c6aad admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:00.822806 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3796a0-7386-4ad4-ad90-76b31b9c6aad admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:00.823095 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3796a0-7386-4ad4-ad90-76b31b9c6aad admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:00.823852 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3796a0-7386-4ad4-ad90-76b31b9c6aad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:00.824018 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 985/1967] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:44:00 2022] POST /identity/v3/users => generated 288 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:44:00.826261 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3796a0-7386-4ad4-ad90-76b31b9c6aad admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:00.828609 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f67931b-fa41-4443-84ce-07a8f8f0cbd5 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:00.839184 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f67931b-fa41-4443-84ce-07a8f8f0cbd5 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:00.845782 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8d39aa-ca3f-4aec-adc8-31c395d09d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:00.846484 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d39aa-ca3f-4aec-adc8-31c395d09d1f admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:00.846844 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d39aa-ca3f-4aec-adc8-31c395d09d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:00.847066 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d39aa-ca3f-4aec-adc8-31c395d09d1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:00.847820 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8d39aa-ca3f-4aec-adc8-31c395d09d1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:00.849533 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8d39aa-ca3f-4aec-adc8-31c395d09d1f admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:00.860520 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 986/1968] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:44:00.861221 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 983/1969] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:44:00 2022] POST /identity/v3/projects => generated 361 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:44:00.864502 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8d39aa-ca3f-4aec-adc8-31c395d09d1f admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:00.867602 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3796a0-7386-4ad4-ad90-76b31b9c6aad admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:00.875196 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8d39aa-ca3f-4aec-adc8-31c395d09d1f admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:00.933307 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:00.979644 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d61917-24e9-47e0-b3a2-92b8232c9f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:00.980719 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d61917-24e9-47e0-b3a2-92b8232c9f7a admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:00.981388 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d61917-24e9-47e0-b3a2-92b8232c9f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:00.981944 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d61917-24e9-47e0-b3a2-92b8232c9f7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:00.982463 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3796a0-7386-4ad4-ad90-76b31b9c6aad admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:00.983108 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d61917-24e9-47e0-b3a2-92b8232c9f7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:00.986216 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d61917-24e9-47e0-b3a2-92b8232c9f7a admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:00.988211 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f469f6a-8fdb-4741-8187-99ccf68159b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:00.988934 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f469f6a-8fdb-4741-8187-99ccf68159b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:00.989230 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f469f6a-8fdb-4741-8187-99ccf68159b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:00.989544 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f469f6a-8fdb-4741-8187-99ccf68159b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:00.990233 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f469f6a-8fdb-4741-8187-99ccf68159b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:00.992194 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f469f6a-8fdb-4741-8187-99ccf68159b4 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:01.000435 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 987/1970] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:44:01.005096 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d61917-24e9-47e0-b3a2-92b8232c9f7a admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:01.015904 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d61917-24e9-47e0-b3a2-92b8232c9f7a admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:01.021096 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a4638f-9b93-47a2-8beb-7afb7515066d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:01.021355 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 984/1971] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:44:00 2022] POST /identity/v3/users => generated 281 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:44:01.021951 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4638f-9b93-47a2-8beb-7afb7515066d admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:01.022242 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4638f-9b93-47a2-8beb-7afb7515066d admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:01.022531 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4638f-9b93-47a2-8beb-7afb7515066d admin admin] PATH_INFO: `/v3/projects/264dc81c40134504a960161f02d76df8/users/e607d36d04fa42439226e33517322783/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:01.025186 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f469f6a-8fdb-4741-8187-99ccf68159b4 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:01.037030 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f469f6a-8fdb-4741-8187-99ccf68159b4 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:01.042304 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dc2b3e-3993-410e-9f38-f1346dfe351d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:01.043122 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc2b3e-3993-410e-9f38-f1346dfe351d admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:01.043663 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc2b3e-3993-410e-9f38-f1346dfe351d admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:01.043663 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc2b3e-3993-410e-9f38-f1346dfe351d admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:01.044411 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dc2b3e-3993-410e-9f38-f1346dfe351d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:01.046068 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dc2b3e-3993-410e-9f38-f1346dfe351d admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:01.053574 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a4638f-9b93-47a2-8beb-7afb7515066d admin admin] RBAC: Authorizing `identity:create_grant(project_id=264dc81c40134504a960161f02d76df8, user_id=e607d36d04fa42439226e33517322783, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:01.056627 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a4638f-9b93-47a2-8beb-7afb7515066d admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:01.058531 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 985/1972] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:44:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:44:01.063294 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dc2b3e-3993-410e-9f38-f1346dfe351d admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:01.074701 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dc2b3e-3993-410e-9f38-f1346dfe351d admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:01.097255 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 988/1973] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:44:01 2022] PUT /identity/v3/projects/264dc81c40134504a960161f02d76df8/users/e607d36d04fa42439226e33517322783/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:44:01.099875 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d2b78-706f-4df0-a723-ad08fcc5a66b None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:01.100338 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d2b78-706f-4df0-a723-ad08fcc5a66b None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:01.101522 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d2b78-706f-4df0-a723-ad08fcc5a66b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:01.158267 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-9d9d2b78-706f-4df0-a723-ad08fcc5a66b None None] MFA Rules not processed for user `e607d36d04fa42439226e33517322783`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:01.159554 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-9d9d2b78-706f-4df0-a723-ad08fcc5a66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:01.165980 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe60bd2-4cdf-4e59-ac12-121fe95ca713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:01.170101 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe60bd2-4cdf-4e59-ac12-121fe95ca713 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:01.170407 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe60bd2-4cdf-4e59-ac12-121fe95ca713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:01.170703 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe60bd2-4cdf-4e59-ac12-121fe95ca713 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:01.174257 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe60bd2-4cdf-4e59-ac12-121fe95ca713 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:01.177865 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe60bd2-4cdf-4e59-ac12-121fe95ca713 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:01.197591 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 986/1974] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:44:01 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:44:01.202030 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe60bd2-4cdf-4e59-ac12-121fe95ca713 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:01.214055 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe60bd2-4cdf-4e59-ac12-121fe95ca713 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:01.220738 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc608fd6-b667-49e0-9c25-98c03923df54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:01.222095 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc608fd6-b667-49e0-9c25-98c03923df54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:01.222392 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc608fd6-b667-49e0-9c25-98c03923df54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:01.222832 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc608fd6-b667-49e0-9c25-98c03923df54 admin admin] PATH_INFO: `/v3/projects/fdee008a50094e2d9a9eecd2fa1f7a64/users/de350a1d6fed4c46b4993d4733d16944/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:01.244029 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc608fd6-b667-49e0-9c25-98c03923df54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdee008a50094e2d9a9eecd2fa1f7a64, user_id=de350a1d6fed4c46b4993d4733d16944, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:01.247390 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc608fd6-b667-49e0-9c25-98c03923df54 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:01.247519 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 989/1975] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:44:01.252138 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9d2b78-706f-4df0-a723-ad08fcc5a66b None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:01.264945 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9d2b78-706f-4df0-a723-ad08fcc5a66b None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:01.277009 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 987/1976] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:44:01 2022] PUT /identity/v3/projects/fdee008a50094e2d9a9eecd2fa1f7a64/users/de350a1d6fed4c46b4993d4733d16944/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:44:01.277208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:01.281494 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221c71b-bf85-4097-8986-ac5791ca4b14 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:01.281852 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221c71b-bf85-4097-8986-ac5791ca4b14 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:01.282137 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221c71b-bf85-4097-8986-ac5791ca4b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:01.318015 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-7221c71b-bf85-4097-8986-ac5791ca4b14 None None] MFA Rules not processed for user `de350a1d6fed4c46b4993d4733d16944`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:01.320263 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-7221c71b-bf85-4097-8986-ac5791ca4b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:01.340087 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59757ab3-3938-42aa-b632-2be2622914ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:01.340887 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59757ab3-3938-42aa-b632-2be2622914ba admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:01.341203 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59757ab3-3938-42aa-b632-2be2622914ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:01.341491 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59757ab3-3938-42aa-b632-2be2622914ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:01.342355 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59757ab3-3938-42aa-b632-2be2622914ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:01.345026 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59757ab3-3938-42aa-b632-2be2622914ba admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:01.381451 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 990/1977] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:44:01 2022] POST /identity/v3/projects => generated 377 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:44:01.384552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59757ab3-3938-42aa-b632-2be2622914ba admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:01.441227 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 988/1978] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:01 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:44:01.473110 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7221c71b-bf85-4097-8986-ac5791ca4b14 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:01.474782 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-7221c71b-bf85-4097-8986-ac5791ca4b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:01.494138 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:01.526083 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59757ab3-3938-42aa-b632-2be2622914ba admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:01.533090 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed774e2-c7fe-40dd-afdd-784ad5330b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:01.534107 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed774e2-c7fe-40dd-afdd-784ad5330b19 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:01.534593 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed774e2-c7fe-40dd-afdd-784ad5330b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:01.535046 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed774e2-c7fe-40dd-afdd-784ad5330b19 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:01.536088 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed774e2-c7fe-40dd-afdd-784ad5330b19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:01.538860 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed774e2-c7fe-40dd-afdd-784ad5330b19 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:01.578656 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 991/1979] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:44:01 2022] POST /identity/v3/users => generated 289 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:44:01.579912 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed774e2-c7fe-40dd-afdd-784ad5330b19 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:01.588721 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7221c71b-bf85-4097-8986-ac5791ca4b14 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:01.596469 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed774e2-c7fe-40dd-afdd-784ad5330b19 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:01.602038 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e4fb7f-bdb2-4006-86db-793f44791fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:01.603017 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4fb7f-bdb2-4006-86db-793f44791fd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:01.603487 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4fb7f-bdb2-4006-86db-793f44791fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:01.603917 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4fb7f-bdb2-4006-86db-793f44791fd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:01.604809 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e4fb7f-bdb2-4006-86db-793f44791fd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:01.606714 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e4fb7f-bdb2-4006-86db-793f44791fd6 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:01.618546 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 992/1980] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:44:01 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:44:01.623750 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e4fb7f-bdb2-4006-86db-793f44791fd6 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:01.639430 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e4fb7f-bdb2-4006-86db-793f44791fd6 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:01.654209 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d00a71-bb2c-4931-860e-4df4d056e1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:01.654912 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d00a71-bb2c-4931-860e-4df4d056e1ec admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:01.655340 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d00a71-bb2c-4931-860e-4df4d056e1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:01.656261 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d00a71-bb2c-4931-860e-4df4d056e1ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:01.657071 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d00a71-bb2c-4931-860e-4df4d056e1ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:01.657292 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd84a7b-d8ce-46fa-abf5-ca3f1a581676 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:01.657960 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd84a7b-d8ce-46fa-abf5-ca3f1a581676 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:01.658324 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd84a7b-d8ce-46fa-abf5-ca3f1a581676 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:01.658638 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd84a7b-d8ce-46fa-abf5-ca3f1a581676 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:01.659342 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d00a71-bb2c-4931-860e-4df4d056e1ec admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:01.673221 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 993/1981] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:44:01 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:44:01.675667 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d00a71-bb2c-4931-860e-4df4d056e1ec admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:01.688678 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d00a71-bb2c-4931-860e-4df4d056e1ec admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:01.696977 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d713fb6-5c6d-4bc9-b17b-78f964f0ae78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:01.698841 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d713fb6-5c6d-4bc9-b17b-78f964f0ae78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:01.699128 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d713fb6-5c6d-4bc9-b17b-78f964f0ae78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:01.699413 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d713fb6-5c6d-4bc9-b17b-78f964f0ae78 admin admin] PATH_INFO: `/v3/projects/0753ff14582b4b3e832c29c0459d08dd/users/f2e35eaaf9e045a79bb28bae2f314bff/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:01.723036 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d713fb6-5c6d-4bc9-b17b-78f964f0ae78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0753ff14582b4b3e832c29c0459d08dd, user_id=f2e35eaaf9e045a79bb28bae2f314bff, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:01.725567 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd84a7b-d8ce-46fa-abf5-ca3f1a581676 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:01.726481 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d713fb6-5c6d-4bc9-b17b-78f964f0ae78 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:01.727221 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd84a7b-d8ce-46fa-abf5-ca3f1a581676 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:01.739272 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 989/1982] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:44:01 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 16:44:01.746058 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 994/1983] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:44:01 2022] PUT /identity/v3/projects/0753ff14582b4b3e832c29c0459d08dd/users/f2e35eaaf9e045a79bb28bae2f314bff/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:01.760468 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd84a7b-d8ce-46fa-abf5-ca3f1a581676 service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:01.761994 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278f6e4f-3e90-47f8-a58d-ba90d957b32a None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:01.762310 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278f6e4f-3e90-47f8-a58d-ba90d957b32a None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:01.762569 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278f6e4f-3e90-47f8-a58d-ba90d957b32a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:01.777415 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd84a7b-d8ce-46fa-abf5-ca3f1a581676 service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:01.799909 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-278f6e4f-3e90-47f8-a58d-ba90d957b32a None None] MFA Rules not processed for user `f2e35eaaf9e045a79bb28bae2f314bff`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:01.801913 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-278f6e4f-3e90-47f8-a58d-ba90d957b32a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:01.845828 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:01.850049 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd9a62-5178-4e14-9e83-a16dda571ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:01.851188 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd9a62-5178-4e14-9e83-a16dda571ea4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:01.851662 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd9a62-5178-4e14-9e83-a16dda571ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:01.853874 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd9a62-5178-4e14-9e83-a16dda571ea4 admin admin] PATH_INFO: `/v3/users/de350a1d6fed4c46b4993d4733d16944` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:01.853874 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fd9a62-5178-4e14-9e83-a16dda571ea4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de350a1d6fed4c46b4993d4733d16944)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:01.855906 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fd9a62-5178-4e14-9e83-a16dda571ea4 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:01.898869 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 995/1984] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:44:01.903261 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278f6e4f-3e90-47f8-a58d-ba90d957b32a None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:01.915433 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278f6e4f-3e90-47f8-a58d-ba90d957b32a None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:01.918182 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:01.922486 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f421515-b74e-4f66-926c-0dd3f612e0fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:01.923170 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f421515-b74e-4f66-926c-0dd3f612e0fa admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:01.923459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f421515-b74e-4f66-926c-0dd3f612e0fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:01.923763 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f421515-b74e-4f66-926c-0dd3f612e0fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:01.924449 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f421515-b74e-4f66-926c-0dd3f612e0fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:01.926461 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f421515-b74e-4f66-926c-0dd3f612e0fa admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:01.952536 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f7fd9a62-5178-4e14-9e83-a16dda571ea4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de350a1d6fed4c46b4993d4733d16944'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:01.954080 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f7fd9a62-5178-4e14-9e83-a16dda571ea4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de350a1d6fed4c46b4993d4733d16944'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:01.958898 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 996/1985] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:44:01 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:44:01.963174 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f421515-b74e-4f66-926c-0dd3f612e0fa admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:01.965985 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f7fd9a62-5178-4e14-9e83-a16dda571ea4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de350a1d6fed4c46b4993d4733d16944'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:01.979625 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f7fd9a62-5178-4e14-9e83-a16dda571ea4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de350a1d6fed4c46b4993d4733d16944'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:01.980578 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f7fd9a62-5178-4e14-9e83-a16dda571ea4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de350a1d6fed4c46b4993d4733d16944'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:01.993944 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 990/1986] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:44:01 2022] DELETE /identity/v3/users/de350a1d6fed4c46b4993d4733d16944 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:01.998434 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa81ca-96aa-4dde-b1f9-795eab8bdf80 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:01.998733 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa81ca-96aa-4dde-b1f9-795eab8bdf80 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:01.999173 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa81ca-96aa-4dde-b1f9-795eab8bdf80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:02.019199 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-9daa81ca-96aa-4dde-b1f9-795eab8bdf80 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:02.020827 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-9daa81ca-96aa-4dde-b1f9-795eab8bdf80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:02.046031 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f421515-b74e-4f66-926c-0dd3f612e0fa admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:02.087746 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 991/1987] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:44:02.099635 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daa81ca-96aa-4dde-b1f9-795eab8bdf80 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:02.101954 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-9daa81ca-96aa-4dde-b1f9-795eab8bdf80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:02.113326 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d602b65b-5415-420c-b13b-becfe49cf52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:02.114049 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602b65b-5415-420c-b13b-becfe49cf52f admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:02.114334 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602b65b-5415-420c-b13b-becfe49cf52f admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:02.114612 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602b65b-5415-420c-b13b-becfe49cf52f admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:02.115318 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d602b65b-5415-420c-b13b-becfe49cf52f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:02.117292 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d602b65b-5415-420c-b13b-becfe49cf52f admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:02.146918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 997/1988] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:44:01 2022] POST /identity/v3/users => generated 294 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:44:02.150537 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d602b65b-5415-420c-b13b-becfe49cf52f admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:02.152300 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d602b65b-5415-420c-b13b-becfe49cf52f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:02.166739 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d602b65b-5415-420c-b13b-becfe49cf52f admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:02.175871 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe234c84-4b74-469c-9563-0b589255b891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:02.176331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe234c84-4b74-469c-9563-0b589255b891 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:02.176614 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe234c84-4b74-469c-9563-0b589255b891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:02.176898 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe234c84-4b74-469c-9563-0b589255b891 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:02.177845 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe234c84-4b74-469c-9563-0b589255b891 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:02.179579 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe234c84-4b74-469c-9563-0b589255b891 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:02.192143 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 998/1989] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:44:02 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:44:02.193223 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daa81ca-96aa-4dde-b1f9-795eab8bdf80 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:02.196480 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe234c84-4b74-469c-9563-0b589255b891 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:02.198958 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b35d4f8-6624-4e6c-83ef-4a70c634e1e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:02.199867 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35d4f8-6624-4e6c-83ef-4a70c634e1e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:02.200253 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35d4f8-6624-4e6c-83ef-4a70c634e1e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:02.200636 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35d4f8-6624-4e6c-83ef-4a70c634e1e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:02.211827 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b35d4f8-6624-4e6c-83ef-4a70c634e1e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:02.212011 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe234c84-4b74-469c-9563-0b589255b891 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:02.213556 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b35d4f8-6624-4e6c-83ef-4a70c634e1e8 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:02.217447 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d616c-944f-42c8-a841-1d5d8be9dce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:02.218508 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d616c-944f-42c8-a841-1d5d8be9dce2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:02.218940 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d616c-944f-42c8-a841-1d5d8be9dce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:02.219357 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d616c-944f-42c8-a841-1d5d8be9dce2 admin admin] PATH_INFO: `/v3/projects/3f09461186e94571a5e5cd3c43471f51/users/b637fc9d18df41a1aafbcea458c8fc30/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:02.225275 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 992/1990] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:44:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 16:44:02.239524 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9d616c-944f-42c8-a841-1d5d8be9dce2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f09461186e94571a5e5cd3c43471f51, user_id=b637fc9d18df41a1aafbcea458c8fc30, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:02.242361 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9d616c-944f-42c8-a841-1d5d8be9dce2 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:02.263354 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 999/1991] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:44:02 2022] PUT /identity/v3/projects/3f09461186e94571a5e5cd3c43471f51/users/b637fc9d18df41a1aafbcea458c8fc30/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:44:02.267346 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b35d4f8-6624-4e6c-83ef-4a70c634e1e8 service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:02.278538 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b35d4f8-6624-4e6c-83ef-4a70c634e1e8 service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:02.336413 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:02.354365 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95ce0cd-a5b1-4895-bfe3-d536e6f596ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:02.355208 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ce0cd-a5b1-4895-bfe3-d536e6f596ab admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:02.355393 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ce0cd-a5b1-4895-bfe3-d536e6f596ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:02.355669 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ce0cd-a5b1-4895-bfe3-d536e6f596ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:02.357064 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95ce0cd-a5b1-4895-bfe3-d536e6f596ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:02.359274 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95ce0cd-a5b1-4895-bfe3-d536e6f596ab admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:02.373820 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 993/1992] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:44:02 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:44:02.378306 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d616c-944f-42c8-a841-1d5d8be9dce2 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:02.390412 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d616c-944f-42c8-a841-1d5d8be9dce2 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:02.396597 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f8bdb0-a41c-4b7c-93b0-aaf2353b4b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:02.397736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f8bdb0-a41c-4b7c-93b0-aaf2353b4b6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:02.398121 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f8bdb0-a41c-4b7c-93b0-aaf2353b4b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:02.398503 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f8bdb0-a41c-4b7c-93b0-aaf2353b4b6d admin admin] PATH_INFO: `/v3/projects/3f09461186e94571a5e5cd3c43471f51/users/b637fc9d18df41a1aafbcea458c8fc30/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:02.400984 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f8bdb0-a41c-4b7c-93b0-aaf2353b4b6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f09461186e94571a5e5cd3c43471f51, user_id=b637fc9d18df41a1aafbcea458c8fc30, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:02.404391 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f8bdb0-a41c-4b7c-93b0-aaf2353b4b6d admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:02.408783 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:02.420152 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1000/1993] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:44:02 2022] PUT /identity/v3/projects/3f09461186e94571a5e5cd3c43471f51/users/b637fc9d18df41a1aafbcea458c8fc30/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:44:02.431156 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2139b09-89e2-426d-b87b-0a9359e9d836 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:02.431451 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2139b09-89e2-426d-b87b-0a9359e9d836 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:02.431736 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2139b09-89e2-426d-b87b-0a9359e9d836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:02.468544 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-e2139b09-89e2-426d-b87b-0a9359e9d836 None None] MFA Rules not processed for user `b637fc9d18df41a1aafbcea458c8fc30`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:02.471139 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-e2139b09-89e2-426d-b87b-0a9359e9d836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:02.582226 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 994/1994] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:02 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:44:02.594737 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f8bdb0-a41c-4b7c-93b0-aaf2353b4b6d admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:02.596512 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e8f8bdb0-a41c-4b7c-93b0-aaf2353b4b6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:02.667338 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2139b09-89e2-426d-b87b-0a9359e9d836 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:02.669268 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-e2139b09-89e2-426d-b87b-0a9359e9d836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:02.670682 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f8bdb0-a41c-4b7c-93b0-aaf2353b4b6d admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:02.674683 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3ca74-3718-41b2-a45d-5cf7a7cc7829 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:02.675357 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3ca74-3718-41b2-a45d-5cf7a7cc7829 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:02.675645 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3ca74-3718-41b2-a45d-5cf7a7cc7829 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:02.675933 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3ca74-3718-41b2-a45d-5cf7a7cc7829 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:02.677863 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e4f3ca74-3718-41b2-a45d-5cf7a7cc7829 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:02.756889 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f3ca74-3718-41b2-a45d-5cf7a7cc7829 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:02.758787 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f3ca74-3718-41b2-a45d-5cf7a7cc7829 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:02.761609 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2139b09-89e2-426d-b87b-0a9359e9d836 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:02.768113 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c7af4-a073-4aad-a460-ff99c5a1eb6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:02.768988 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c7af4-a073-4aad-a460-ff99c5a1eb6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:02.769307 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c7af4-a073-4aad-a460-ff99c5a1eb6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:02.769588 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c7af4-a073-4aad-a460-ff99c5a1eb6b admin admin] PATH_INFO: `/v3/projects/fdee008a50094e2d9a9eecd2fa1f7a64` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:02.770768 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1001/1995] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:44:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:44:02.773529 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047c7af4-a073-4aad-a460-ff99c5a1eb6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdee008a50094e2d9a9eecd2fa1f7a64)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:02.776104 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047c7af4-a073-4aad-a460-ff99c5a1eb6b admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:02.834881 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-047c7af4-a073-4aad-a460-ff99c5a1eb6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdee008a50094e2d9a9eecd2fa1f7a64', 'request_id': 'req-047c7af4-a073-4aad-a460-ff99c5a1eb6b'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:02.843249 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-047c7af4-a073-4aad-a460-ff99c5a1eb6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdee008a50094e2d9a9eecd2fa1f7a64', 'request_id': 'req-047c7af4-a073-4aad-a460-ff99c5a1eb6b'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:02.851354 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-047c7af4-a073-4aad-a460-ff99c5a1eb6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdee008a50094e2d9a9eecd2fa1f7a64', 'request_id': 'req-047c7af4-a073-4aad-a460-ff99c5a1eb6b'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:02.862058 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:02.867445 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-047c7af4-a073-4aad-a460-ff99c5a1eb6b admin admin] The token cache is being invalidate because project fdee008a50094e2d9a9eecd2fa1f7a64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:44:02.868055 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-047c7af4-a073-4aad-a460-ff99c5a1eb6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:02.871208 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-047c7af4-a073-4aad-a460-ff99c5a1eb6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:02.877272 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 995/1996] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:44:02 2022] DELETE /identity/v3/projects/fdee008a50094e2d9a9eecd2fa1f7a64 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:44:02.878823 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc10caf-c729-41c4-a30d-10f189492b82 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:02.879131 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc10caf-c729-41c4-a30d-10f189492b82 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:02.879402 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc10caf-c729-41c4-a30d-10f189492b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:02.905341 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-1bc10caf-c729-41c4-a30d-10f189492b82 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:02.907385 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-1bc10caf-c729-41c4-a30d-10f189492b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:03.004579 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1002/1997] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:44:03.006746 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a7d5e-1207-4397-944a-524eadd98a0f None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:03.007330 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a7d5e-1207-4397-944a-524eadd98a0f None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:03.007774 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a7d5e-1207-4397-944a-524eadd98a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:03.037425 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-880a7d5e-1207-4397-944a-524eadd98a0f None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:03.045400 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-880a7d5e-1207-4397-944a-524eadd98a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:03.057628 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 996/1998] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:44:03.061278 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc10caf-c729-41c4-a30d-10f189492b82 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:03.084869 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc10caf-c729-41c4-a30d-10f189492b82 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:03.094513 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3ddf9d-c80c-4085-8d2c-20963d64f0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:03.095199 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ddf9d-c80c-4085-8d2c-20963d64f0d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:03.095619 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ddf9d-c80c-4085-8d2c-20963d64f0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:03.096068 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ddf9d-c80c-4085-8d2c-20963d64f0d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:03.096878 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3ddf9d-c80c-4085-8d2c-20963d64f0d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:44:03.097228 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3ddf9d-c80c-4085-8d2c-20963d64f0d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:03.099063 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3ddf9d-c80c-4085-8d2c-20963d64f0d2 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:03.117368 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1003/1999] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:44:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:44:03.117849 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a7d5e-1207-4397-944a-524eadd98a0f None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:03.137666 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a7d5e-1207-4397-944a-524eadd98a0f None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:03.152759 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a48fb6-5474-4037-bf22-21884ea1dcec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:03.153471 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a48fb6-5474-4037-bf22-21884ea1dcec admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:03.153707 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a48fb6-5474-4037-bf22-21884ea1dcec admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:03.153987 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a48fb6-5474-4037-bf22-21884ea1dcec admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:03.154998 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a48fb6-5474-4037-bf22-21884ea1dcec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:03.156744 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a48fb6-5474-4037-bf22-21884ea1dcec admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:03.185838 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 997/2000] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:44:03 2022] POST /identity/v3/projects => generated 375 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:44:03.190639 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3ddf9d-c80c-4085-8d2c-20963d64f0d2 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:03.200798 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3ddf9d-c80c-4085-8d2c-20963d64f0d2 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:03.236610 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:03.261682 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dfc042-f9d0-49f1-99df-808c356c3712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:03.262355 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dfc042-f9d0-49f1-99df-808c356c3712 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:03.262638 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dfc042-f9d0-49f1-99df-808c356c3712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:03.262917 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dfc042-f9d0-49f1-99df-808c356c3712 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:03.263603 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dfc042-f9d0-49f1-99df-808c356c3712 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:03.265598 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dfc042-f9d0-49f1-99df-808c356c3712 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:03.290254 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1004/2001] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:44:03 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:44:03.294136 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a48fb6-5474-4037-bf22-21884ea1dcec admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:03.305956 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a48fb6-5474-4037-bf22-21884ea1dcec admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:03.311477 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fd59f3-66be-4b53-be84-f825b098da5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:03.312841 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd59f3-66be-4b53-be84-f825b098da5d admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:03.313133 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd59f3-66be-4b53-be84-f825b098da5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:03.313535 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd59f3-66be-4b53-be84-f825b098da5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:03.314402 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fd59f3-66be-4b53-be84-f825b098da5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:03.316162 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fd59f3-66be-4b53-be84-f825b098da5d admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:03.330269 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 998/2002] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:44:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:44:03.332054 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dfc042-f9d0-49f1-99df-808c356c3712 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:03.342806 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dfc042-f9d0-49f1-99df-808c356c3712 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:03.348417 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c3a13-6281-4059-8896-ecb09f62685c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:03.349100 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c3a13-6281-4059-8896-ecb09f62685c admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:03.349424 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c3a13-6281-4059-8896-ecb09f62685c admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:03.349711 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c3a13-6281-4059-8896-ecb09f62685c admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:03.350447 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9c3a13-6281-4059-8896-ecb09f62685c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:03.352077 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9c3a13-6281-4059-8896-ecb09f62685c admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:03.366911 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1005/2003] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:44:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:44:03.368200 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fd59f3-66be-4b53-be84-f825b098da5d admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:03.378377 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fd59f3-66be-4b53-be84-f825b098da5d admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:03.383796 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b596c8b1-2826-4378-8d6a-6aa7d0e5d8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:03.384252 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b596c8b1-2826-4378-8d6a-6aa7d0e5d8ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:03.384542 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b596c8b1-2826-4378-8d6a-6aa7d0e5d8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:03.384819 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b596c8b1-2826-4378-8d6a-6aa7d0e5d8ef admin admin] PATH_INFO: `/v3/projects/73722a665d224bf8b144b639a0189a2a/users/3402e4bda8f14a1bb6b106a58b828710/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:03.400084 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b596c8b1-2826-4378-8d6a-6aa7d0e5d8ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=73722a665d224bf8b144b639a0189a2a, user_id=3402e4bda8f14a1bb6b106a58b828710, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:03.402794 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b596c8b1-2826-4378-8d6a-6aa7d0e5d8ef admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:03.417297 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 999/2004] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:44:03 2022] PUT /identity/v3/projects/73722a665d224bf8b144b639a0189a2a/users/3402e4bda8f14a1bb6b106a58b828710/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:44:03.428237 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafaaaef-fcf8-480a-ad07-cd925e808286 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:03.428539 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafaaaef-fcf8-480a-ad07-cd925e808286 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:03.428825 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafaaaef-fcf8-480a-ad07-cd925e808286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:03.456014 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-fafaaaef-fcf8-480a-ad07-cd925e808286 None None] MFA Rules not processed for user `3402e4bda8f14a1bb6b106a58b828710`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:03.457323 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-fafaaaef-fcf8-480a-ad07-cd925e808286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:03.523912 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1006/2005] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:44:03.536203 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b596c8b1-2826-4378-8d6a-6aa7d0e5d8ef admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:03.537964 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-b596c8b1-2826-4378-8d6a-6aa7d0e5d8ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:03.600468 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b596c8b1-2826-4378-8d6a-6aa7d0e5d8ef admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:03.604238 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce987428-9b8b-4421-a8d8-3f08bc74e53c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:03.604909 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce987428-9b8b-4421-a8d8-3f08bc74e53c service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:03.605227 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce987428-9b8b-4421-a8d8-3f08bc74e53c service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:03.605515 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce987428-9b8b-4421-a8d8-3f08bc74e53c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:03.615486 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce987428-9b8b-4421-a8d8-3f08bc74e53c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:03.616967 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce987428-9b8b-4421-a8d8-3f08bc74e53c service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:03.626450 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1000/2006] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:44:03 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:44:09.666226 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafaaaef-fcf8-480a-ad07-cd925e808286 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:09.677552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafaaaef-fcf8-480a-ad07-cd925e808286 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:09.681850 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16466baf-de9d-4b95-b424-8894dcd03d65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:09.682534 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16466baf-de9d-4b95-b424-8894dcd03d65 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:09.682848 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16466baf-de9d-4b95-b424-8894dcd03d65 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:09.683206 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16466baf-de9d-4b95-b424-8894dcd03d65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:09.685282 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-16466baf-de9d-4b95-b424-8894dcd03d65 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:09.754660 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16466baf-de9d-4b95-b424-8894dcd03d65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:09.756478 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16466baf-de9d-4b95-b424-8894dcd03d65 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:09.766208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1007/2007] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:44:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:44:10.605472 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce987428-9b8b-4421-a8d8-3f08bc74e53c service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:10.607388 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ce987428-9b8b-4421-a8d8-3f08bc74e53c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:10.622034 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce987428-9b8b-4421-a8d8-3f08bc74e53c service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:10.626750 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d711fe46-85d2-4076-a3b5-f947d43c4f23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:10.627493 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711fe46-85d2-4076-a3b5-f947d43c4f23 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:10.627865 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711fe46-85d2-4076-a3b5-f947d43c4f23 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:10.628211 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711fe46-85d2-4076-a3b5-f947d43c4f23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:10.630288 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-d711fe46-85d2-4076-a3b5-f947d43c4f23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:10.732634 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d711fe46-85d2-4076-a3b5-f947d43c4f23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:10.734913 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d711fe46-85d2-4076-a3b5-f947d43c4f23 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:10.746995 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1001/2008] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:44:10 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:44:11.643244 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16466baf-de9d-4b95-b424-8894dcd03d65 service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:11.644903 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-16466baf-de9d-4b95-b424-8894dcd03d65 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:11.695790 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:11.716776 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16466baf-de9d-4b95-b424-8894dcd03d65 service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:11.721487 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5924f13d-9904-4752-b8c7-cb662fadd8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:11.722371 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924f13d-9904-4752-b8c7-cb662fadd8e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:11.722659 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924f13d-9904-4752-b8c7-cb662fadd8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:11.722941 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924f13d-9904-4752-b8c7-cb662fadd8e2 admin admin] PATH_INFO: `/v3/users/d32b6d161b8a4af48231d6afebdbd7e4` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:11.724161 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5924f13d-9904-4752-b8c7-cb662fadd8e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d32b6d161b8a4af48231d6afebdbd7e4)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:11.726189 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5924f13d-9904-4752-b8c7-cb662fadd8e2 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:11.775232 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-5924f13d-9904-4752-b8c7-cb662fadd8e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd32b6d161b8a4af48231d6afebdbd7e4'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:11.775916 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-5924f13d-9904-4752-b8c7-cb662fadd8e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd32b6d161b8a4af48231d6afebdbd7e4'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:11.776533 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-5924f13d-9904-4752-b8c7-cb662fadd8e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd32b6d161b8a4af48231d6afebdbd7e4'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:11.784151 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-5924f13d-9904-4752-b8c7-cb662fadd8e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd32b6d161b8a4af48231d6afebdbd7e4'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:11.796647 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-5924f13d-9904-4752-b8c7-cb662fadd8e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd32b6d161b8a4af48231d6afebdbd7e4'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:11.812990 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1008/2009] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:44:11 2022] DELETE /identity/v3/users/d32b6d161b8a4af48231d6afebdbd7e4 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:11.815075 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8590069-41bb-4726-93ca-965829ac43d6 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:11.815549 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8590069-41bb-4726-93ca-965829ac43d6 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:11.815811 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8590069-41bb-4726-93ca-965829ac43d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:11.833993 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-c8590069-41bb-4726-93ca-965829ac43d6 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:11.835228 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-c8590069-41bb-4726-93ca-965829ac43d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:11.862897 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:11.907254 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1002/2010] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:44:11.916891 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5924f13d-9904-4752-b8c7-cb662fadd8e2 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:11.918473 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-5924f13d-9904-4752-b8c7-cb662fadd8e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:12.009627 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5924f13d-9904-4752-b8c7-cb662fadd8e2 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:12.013530 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45574df-4024-4958-8087-9b0ff3445e65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:12.014191 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45574df-4024-4958-8087-9b0ff3445e65 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:12.014477 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45574df-4024-4958-8087-9b0ff3445e65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:12.014763 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45574df-4024-4958-8087-9b0ff3445e65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:12.024286 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45574df-4024-4958-8087-9b0ff3445e65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:12.025774 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45574df-4024-4958-8087-9b0ff3445e65 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:12.035469 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1009/2011] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:44:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:44:12.321838 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8590069-41bb-4726-93ca-965829ac43d6 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:12.323591 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-c8590069-41bb-4726-93ca-965829ac43d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:12.349560 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8590069-41bb-4726-93ca-965829ac43d6 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:12.355416 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca393ee8-5c66-4c31-8cc5-2af7444f09b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:12.356583 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca393ee8-5c66-4c31-8cc5-2af7444f09b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:12.356973 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca393ee8-5c66-4c31-8cc5-2af7444f09b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:12.357399 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca393ee8-5c66-4c31-8cc5-2af7444f09b1 admin admin] PATH_INFO: `/v3/projects/3d47fa99bb924764bd6c15da44eb2a52` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:12.359695 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca393ee8-5c66-4c31-8cc5-2af7444f09b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d47fa99bb924764bd6c15da44eb2a52)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:12.362464 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca393ee8-5c66-4c31-8cc5-2af7444f09b1 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:12.419073 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-ca393ee8-5c66-4c31-8cc5-2af7444f09b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d47fa99bb924764bd6c15da44eb2a52', 'request_id': 'req-ca393ee8-5c66-4c31-8cc5-2af7444f09b1'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:12.427908 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-ca393ee8-5c66-4c31-8cc5-2af7444f09b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d47fa99bb924764bd6c15da44eb2a52', 'request_id': 'req-ca393ee8-5c66-4c31-8cc5-2af7444f09b1'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:12.435312 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-ca393ee8-5c66-4c31-8cc5-2af7444f09b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d47fa99bb924764bd6c15da44eb2a52', 'request_id': 'req-ca393ee8-5c66-4c31-8cc5-2af7444f09b1'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:12.455839 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-ca393ee8-5c66-4c31-8cc5-2af7444f09b1 admin admin] The token cache is being invalidate because project 3d47fa99bb924764bd6c15da44eb2a52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:44:12.456194 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-ca393ee8-5c66-4c31-8cc5-2af7444f09b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:12.456918 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-ca393ee8-5c66-4c31-8cc5-2af7444f09b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:12.459053 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1003/2012] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:44:12 2022] DELETE /identity/v3/projects/3d47fa99bb924764bd6c15da44eb2a52 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:12.462866 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45574df-4024-4958-8087-9b0ff3445e65 service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:12.475068 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45574df-4024-4958-8087-9b0ff3445e65 service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:12.541070 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75660fc-c7b3-469f-8325-ae3f2d8cc887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:12.542003 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75660fc-c7b3-469f-8325-ae3f2d8cc887 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:12.542294 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75660fc-c7b3-469f-8325-ae3f2d8cc887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:12.542576 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75660fc-c7b3-469f-8325-ae3f2d8cc887 admin admin] PATH_INFO: `/v3/users/06766f4366524f929b2d44704ce2f455` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:12.543829 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75660fc-c7b3-469f-8325-ae3f2d8cc887 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06766f4366524f929b2d44704ce2f455)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:12.545845 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75660fc-c7b3-469f-8325-ae3f2d8cc887 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:12.552713 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:12.598587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-d75660fc-c7b3-469f-8325-ae3f2d8cc887 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06766f4366524f929b2d44704ce2f455'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:12.599288 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-d75660fc-c7b3-469f-8325-ae3f2d8cc887 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06766f4366524f929b2d44704ce2f455'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:12.599920 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-d75660fc-c7b3-469f-8325-ae3f2d8cc887 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06766f4366524f929b2d44704ce2f455'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:12.607327 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-d75660fc-c7b3-469f-8325-ae3f2d8cc887 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06766f4366524f929b2d44704ce2f455'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:12.624199 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-d75660fc-c7b3-469f-8325-ae3f2d8cc887 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06766f4366524f929b2d44704ce2f455'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:12.637772 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1010/2013] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:44:12 2022] DELETE /identity/v3/users/06766f4366524f929b2d44704ce2f455 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:12.938485 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca393ee8-5c66-4c31-8cc5-2af7444f09b1 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:12.940494 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ca393ee8-5c66-4c31-8cc5-2af7444f09b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:12.954625 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca393ee8-5c66-4c31-8cc5-2af7444f09b1 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:13.028607 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ee40f-a472-4d29-842f-223b81ca71aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:13.029523 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ee40f-a472-4d29-842f-223b81ca71aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:13.030115 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ee40f-a472-4d29-842f-223b81ca71aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:13.030346 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ee40f-a472-4d29-842f-223b81ca71aa admin admin] PATH_INFO: `/v3/projects/6a6cc6b3b6734db98c8aa814ec142eae` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:13.032227 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6ee40f-a472-4d29-842f-223b81ca71aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a6cc6b3b6734db98c8aa814ec142eae)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:13.034340 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6ee40f-a472-4d29-842f-223b81ca71aa admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:13.081075 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-2d6ee40f-a472-4d29-842f-223b81ca71aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a6cc6b3b6734db98c8aa814ec142eae', 'request_id': 'req-2d6ee40f-a472-4d29-842f-223b81ca71aa'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:13.085761 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:13.086962 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-2d6ee40f-a472-4d29-842f-223b81ca71aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a6cc6b3b6734db98c8aa814ec142eae', 'request_id': 'req-2d6ee40f-a472-4d29-842f-223b81ca71aa'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:13.095813 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-2d6ee40f-a472-4d29-842f-223b81ca71aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a6cc6b3b6734db98c8aa814ec142eae', 'request_id': 'req-2d6ee40f-a472-4d29-842f-223b81ca71aa'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:13.108741 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-2d6ee40f-a472-4d29-842f-223b81ca71aa admin admin] The token cache is being invalidate because project 6a6cc6b3b6734db98c8aa814ec142eae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:44:13.109035 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-2d6ee40f-a472-4d29-842f-223b81ca71aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:13.109694 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-2d6ee40f-a472-4d29-842f-223b81ca71aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:13.111259 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1004/2014] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:44:12 2022] DELETE /identity/v3/projects/6a6cc6b3b6734db98c8aa814ec142eae => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:13.116240 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88221c1-f0ae-43c4-96fd-a50697684be0 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:13.116694 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88221c1-f0ae-43c4-96fd-a50697684be0 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:13.116813 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88221c1-f0ae-43c4-96fd-a50697684be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:13.135541 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-d88221c1-f0ae-43c4-96fd-a50697684be0 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:13.136747 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d88221c1-f0ae-43c4-96fd-a50697684be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:13.197361 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1011/2015] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:44:13.205605 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c0354-6dbc-4028-ab56-b15cffc4bd09 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:13.205903 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c0354-6dbc-4028-ab56-b15cffc4bd09 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:13.206188 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c0354-6dbc-4028-ab56-b15cffc4bd09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:13.224345 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-aa0c0354-6dbc-4028-ab56-b15cffc4bd09 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:13.225575 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-aa0c0354-6dbc-4028-ab56-b15cffc4bd09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:13.233809 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1005/2016] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:44:13.238071 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88221c1-f0ae-43c4-96fd-a50697684be0 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:13.247741 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88221c1-f0ae-43c4-96fd-a50697684be0 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:13.252859 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb60d0a-631c-4043-b846-b30d25b91c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:13.253577 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb60d0a-631c-4043-b846-b30d25b91c8a admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:13.253868 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb60d0a-631c-4043-b846-b30d25b91c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:13.254153 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb60d0a-631c-4043-b846-b30d25b91c8a admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:13.254891 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb60d0a-631c-4043-b846-b30d25b91c8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:44:13.255209 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb60d0a-631c-4043-b846-b30d25b91c8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:13.257089 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb60d0a-631c-4043-b846-b30d25b91c8a admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:13.269330 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1012/2017] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:44:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:44:13.273546 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0c0354-6dbc-4028-ab56-b15cffc4bd09 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:13.282886 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0c0354-6dbc-4028-ab56-b15cffc4bd09 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:13.287636 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c78a92-2bae-4f32-976f-d8804c29365f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:13.288307 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c78a92-2bae-4f32-976f-d8804c29365f admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:13.288595 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c78a92-2bae-4f32-976f-d8804c29365f admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:13.288880 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c78a92-2bae-4f32-976f-d8804c29365f admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:13.289637 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c78a92-2bae-4f32-976f-d8804c29365f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:13.291553 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c78a92-2bae-4f32-976f-d8804c29365f admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:13.317904 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1006/2018] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:44:13 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:44:13.321886 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb60d0a-631c-4043-b846-b30d25b91c8a admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:13.332424 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb60d0a-631c-4043-b846-b30d25b91c8a admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:13.396205 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da8ce5-7753-40d8-995b-8246520ef827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:13.396875 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da8ce5-7753-40d8-995b-8246520ef827 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:13.397208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da8ce5-7753-40d8-995b-8246520ef827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:13.397490 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da8ce5-7753-40d8-995b-8246520ef827 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:13.398197 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5da8ce5-7753-40d8-995b-8246520ef827 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:13.400162 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5da8ce5-7753-40d8-995b-8246520ef827 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:13.423814 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1013/2019] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:44:13 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:44:13.427916 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c78a92-2bae-4f32-976f-d8804c29365f admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:13.438871 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c78a92-2bae-4f32-976f-d8804c29365f admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:13.444695 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766781cd-f70d-4c0b-a339-95ac4d019871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:13.445394 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766781cd-f70d-4c0b-a339-95ac4d019871 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:13.445681 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766781cd-f70d-4c0b-a339-95ac4d019871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:13.445959 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766781cd-f70d-4c0b-a339-95ac4d019871 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:13.446692 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766781cd-f70d-4c0b-a339-95ac4d019871 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:13.448289 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766781cd-f70d-4c0b-a339-95ac4d019871 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:13.459958 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1007/2020] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:44:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:44:13.464243 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da8ce5-7753-40d8-995b-8246520ef827 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:13.469542 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:13.476801 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da8ce5-7753-40d8-995b-8246520ef827 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:13.482670 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115046f6-392d-4002-894a-0d4ea9c6d4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:13.483349 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115046f6-392d-4002-894a-0d4ea9c6d4fb admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:13.484150 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115046f6-392d-4002-894a-0d4ea9c6d4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:13.484432 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115046f6-392d-4002-894a-0d4ea9c6d4fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:13.485206 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115046f6-392d-4002-894a-0d4ea9c6d4fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:13.486859 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115046f6-392d-4002-894a-0d4ea9c6d4fb admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:13.500106 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1014/2021] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:44:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:44:13.502761 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766781cd-f70d-4c0b-a339-95ac4d019871 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:13.513023 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766781cd-f70d-4c0b-a339-95ac4d019871 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:13.517814 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce911d9-2c1b-4d95-9473-54b9c2fac0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:13.518712 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce911d9-2c1b-4d95-9473-54b9c2fac0ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:13.518994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce911d9-2c1b-4d95-9473-54b9c2fac0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:13.519274 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce911d9-2c1b-4d95-9473-54b9c2fac0ff admin admin] PATH_INFO: `/v3/projects/3bd0d1dadadb480ab97b93fb7a5b2562/users/17c5cbd044fb4fd6a3592a5b75cfc6cc/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:13.537329 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce911d9-2c1b-4d95-9473-54b9c2fac0ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bd0d1dadadb480ab97b93fb7a5b2562, user_id=17c5cbd044fb4fd6a3592a5b75cfc6cc, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:13.540180 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce911d9-2c1b-4d95-9473-54b9c2fac0ff admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:13.555252 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1008/2022] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:44:13 2022] PUT /identity/v3/projects/3bd0d1dadadb480ab97b93fb7a5b2562/users/17c5cbd044fb4fd6a3592a5b75cfc6cc/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:13.568402 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3d66f-96a7-4be0-a2b5-1e4c7ad6f33b None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:13.568801 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3d66f-96a7-4be0-a2b5-1e4c7ad6f33b None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:13.569188 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3d66f-96a7-4be0-a2b5-1e4c7ad6f33b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:13.596527 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-b3c3d66f-96a7-4be0-a2b5-1e4c7ad6f33b None None] MFA Rules not processed for user `17c5cbd044fb4fd6a3592a5b75cfc6cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:13.597790 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b3c3d66f-96a7-4be0-a2b5-1e4c7ad6f33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:13.672420 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1015/2023] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:44:13.683308 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce911d9-2c1b-4d95-9473-54b9c2fac0ff admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:13.685178 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-6ce911d9-2c1b-4d95-9473-54b9c2fac0ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:13.747817 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce911d9-2c1b-4d95-9473-54b9c2fac0ff admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:13.752065 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b33737-8242-412d-83b1-3338c42db7b8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57a5d7151478434f86be085bcc487f0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:13.752718 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b33737-8242-412d-83b1-3338c42db7b8 service glance] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:13.753013 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b33737-8242-412d-83b1-3338c42db7b8 service glance] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:13.753330 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b33737-8242-412d-83b1-3338c42db7b8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:13.761895 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b33737-8242-412d-83b1-3338c42db7b8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:13.763331 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b33737-8242-412d-83b1-3338c42db7b8 service glance] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:13.772293 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1009/2024] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:44:13 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:44:17.726573 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c3d66f-96a7-4be0-a2b5-1e4c7ad6f33b None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:17.728013 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b3c3d66f-96a7-4be0-a2b5-1e4c7ad6f33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:17.801994 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c3d66f-96a7-4be0-a2b5-1e4c7ad6f33b None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:17.807082 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca60e3b-e2f2-4cab-8d3b-2b0472ab5176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:17.807956 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca60e3b-e2f2-4cab-8d3b-2b0472ab5176 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:17.808246 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca60e3b-e2f2-4cab-8d3b-2b0472ab5176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:17.808526 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca60e3b-e2f2-4cab-8d3b-2b0472ab5176 admin admin] PATH_INFO: `/v3/users/3402e4bda8f14a1bb6b106a58b828710` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:17.809771 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca60e3b-e2f2-4cab-8d3b-2b0472ab5176 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3402e4bda8f14a1bb6b106a58b828710)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:17.811753 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca60e3b-e2f2-4cab-8d3b-2b0472ab5176 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:17.859827 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-4ca60e3b-e2f2-4cab-8d3b-2b0472ab5176 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3402e4bda8f14a1bb6b106a58b828710'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:17.860564 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-4ca60e3b-e2f2-4cab-8d3b-2b0472ab5176 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3402e4bda8f14a1bb6b106a58b828710'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:17.861439 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-4ca60e3b-e2f2-4cab-8d3b-2b0472ab5176 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3402e4bda8f14a1bb6b106a58b828710'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:17.868989 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-4ca60e3b-e2f2-4cab-8d3b-2b0472ab5176 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3402e4bda8f14a1bb6b106a58b828710'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:17.881284 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-4ca60e3b-e2f2-4cab-8d3b-2b0472ab5176 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3402e4bda8f14a1bb6b106a58b828710'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:17.895205 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:17.901009 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1016/2025] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:44:17 2022] DELETE /identity/v3/users/3402e4bda8f14a1bb6b106a58b828710 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:17.905355 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a6a5d-f09d-42ff-a5b4-73b284c37218 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:17.906356 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a6a5d-f09d-42ff-a5b4-73b284c37218 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:17.906506 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a6a5d-f09d-42ff-a5b4-73b284c37218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:17.924525 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-c75a6a5d-f09d-42ff-a5b4-73b284c37218 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:17.925870 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-c75a6a5d-f09d-42ff-a5b4-73b284c37218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:17.992904 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1010/2026] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:44:18.002623 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca60e3b-e2f2-4cab-8d3b-2b0472ab5176 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:18.004003 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-4ca60e3b-e2f2-4cab-8d3b-2b0472ab5176 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:18.074666 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca60e3b-e2f2-4cab-8d3b-2b0472ab5176 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:18.080471 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c2b83-bae6-4a01-be42-6ec52df5ff45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:18.081342 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c2b83-bae6-4a01-be42-6ec52df5ff45 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:18.081634 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c2b83-bae6-4a01-be42-6ec52df5ff45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:18.081918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c2b83-bae6-4a01-be42-6ec52df5ff45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:18.093739 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007c2b83-bae6-4a01-be42-6ec52df5ff45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:18.095253 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007c2b83-bae6-4a01-be42-6ec52df5ff45 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:18.106197 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1017/2027] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:44:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:44:18.474388 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75a6a5d-f09d-42ff-a5b4-73b284c37218 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:18.478601 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-c75a6a5d-f09d-42ff-a5b4-73b284c37218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:18.534251 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:18.539369 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75a6a5d-f09d-42ff-a5b4-73b284c37218 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:18.583898 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33118343-d91f-4ad2-a27e-2e6548b6e781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:18.585277 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33118343-d91f-4ad2-a27e-2e6548b6e781 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:18.585630 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33118343-d91f-4ad2-a27e-2e6548b6e781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:18.586073 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33118343-d91f-4ad2-a27e-2e6548b6e781 admin admin] PATH_INFO: `/v3/projects/73722a665d224bf8b144b639a0189a2a` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:18.588585 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33118343-d91f-4ad2-a27e-2e6548b6e781 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73722a665d224bf8b144b639a0189a2a)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:18.591427 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33118343-d91f-4ad2-a27e-2e6548b6e781 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:18.668305 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-33118343-d91f-4ad2-a27e-2e6548b6e781 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73722a665d224bf8b144b639a0189a2a', 'request_id': 'req-33118343-d91f-4ad2-a27e-2e6548b6e781'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:18.675626 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-33118343-d91f-4ad2-a27e-2e6548b6e781 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73722a665d224bf8b144b639a0189a2a', 'request_id': 'req-33118343-d91f-4ad2-a27e-2e6548b6e781'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:18.685533 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-33118343-d91f-4ad2-a27e-2e6548b6e781 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73722a665d224bf8b144b639a0189a2a', 'request_id': 'req-33118343-d91f-4ad2-a27e-2e6548b6e781'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:18.704795 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-33118343-d91f-4ad2-a27e-2e6548b6e781 admin admin] The token cache is being invalidate because project 73722a665d224bf8b144b639a0189a2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:44:18.705140 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-33118343-d91f-4ad2-a27e-2e6548b6e781 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:18.705858 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-33118343-d91f-4ad2-a27e-2e6548b6e781 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:18.707778 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1011/2028] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:44:18 2022] DELETE /identity/v3/projects/73722a665d224bf8b144b639a0189a2a => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:18.713279 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b0a57-a2d1-443a-aa27-20f712409460 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:18.713559 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b0a57-a2d1-443a-aa27-20f712409460 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:18.713847 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b0a57-a2d1-443a-aa27-20f712409460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:18.761835 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-a22b0a57-a2d1-443a-aa27-20f712409460 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:18.763106 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-a22b0a57-a2d1-443a-aa27-20f712409460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:18.845834 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1018/2029] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:44:18.855034 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68a4cd-1eb7-45a2-a8b5-d21d8cffa072 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:18.855331 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68a4cd-1eb7-45a2-a8b5-d21d8cffa072 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:18.855657 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68a4cd-1eb7-45a2-a8b5-d21d8cffa072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:18.877534 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-ee68a4cd-1eb7-45a2-a8b5-d21d8cffa072 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:18.878812 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ee68a4cd-1eb7-45a2-a8b5-d21d8cffa072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:18.888613 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1012/2030] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:44:18.893341 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22b0a57-a2d1-443a-aa27-20f712409460 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:18.907852 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22b0a57-a2d1-443a-aa27-20f712409460 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:18.912815 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc53b19-7275-46c6-8ec4-34e35c2eb094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:18.913589 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc53b19-7275-46c6-8ec4-34e35c2eb094 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:18.913928 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc53b19-7275-46c6-8ec4-34e35c2eb094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:18.914188 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc53b19-7275-46c6-8ec4-34e35c2eb094 admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:18.914933 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc53b19-7275-46c6-8ec4-34e35c2eb094 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:44:18.915304 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc53b19-7275-46c6-8ec4-34e35c2eb094 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:18.916983 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc53b19-7275-46c6-8ec4-34e35c2eb094 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:18.941186 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1019/2031] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:44:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:44:18.942906 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68a4cd-1eb7-45a2-a8b5-d21d8cffa072 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:18.952500 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68a4cd-1eb7-45a2-a8b5-d21d8cffa072 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:18.957982 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b4ada5-e9dd-44d7-a34b-86dfdabe3e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:18.958424 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4ada5-e9dd-44d7-a34b-86dfdabe3e52 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:18.958713 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4ada5-e9dd-44d7-a34b-86dfdabe3e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:18.958995 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4ada5-e9dd-44d7-a34b-86dfdabe3e52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:18.960083 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b4ada5-e9dd-44d7-a34b-86dfdabe3e52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:18.962168 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b4ada5-e9dd-44d7-a34b-86dfdabe3e52 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:18.996080 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1013/2032] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:44:18 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:44:19.000980 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc53b19-7275-46c6-8ec4-34e35c2eb094 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:19.011478 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc53b19-7275-46c6-8ec4-34e35c2eb094 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:19.079593 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ebd98b-357e-4c43-b271-6bd5ee572779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:19.080285 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebd98b-357e-4c43-b271-6bd5ee572779 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:19.080574 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebd98b-357e-4c43-b271-6bd5ee572779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:19.080860 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebd98b-357e-4c43-b271-6bd5ee572779 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:19.081622 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ebd98b-357e-4c43-b271-6bd5ee572779 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:19.083601 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ebd98b-357e-4c43-b271-6bd5ee572779 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:19.094726 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:19.107604 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1020/2033] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:44:18 2022] POST /identity/v3/users => generated 291 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:44:19.112233 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b4ada5-e9dd-44d7-a34b-86dfdabe3e52 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:19.123222 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b4ada5-e9dd-44d7-a34b-86dfdabe3e52 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:19.129403 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c3dfaf-b490-4cfe-b0e5-e0f31dc8c681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:19.130805 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c3dfaf-b490-4cfe-b0e5-e0f31dc8c681 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:19.131261 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c3dfaf-b490-4cfe-b0e5-e0f31dc8c681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:19.131732 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c3dfaf-b490-4cfe-b0e5-e0f31dc8c681 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:19.132520 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c3dfaf-b490-4cfe-b0e5-e0f31dc8c681 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:19.134148 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c3dfaf-b490-4cfe-b0e5-e0f31dc8c681 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:19.145576 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1014/2034] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:44:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:44:19.149324 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ebd98b-357e-4c43-b271-6bd5ee572779 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:19.160105 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ebd98b-357e-4c43-b271-6bd5ee572779 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:19.165429 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb7e40-2525-4b96-b63c-93f9a7399e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:19.166114 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb7e40-2525-4b96-b63c-93f9a7399e5a admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:19.166411 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb7e40-2525-4b96-b63c-93f9a7399e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:19.166695 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb7e40-2525-4b96-b63c-93f9a7399e5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:19.167436 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbb7e40-2525-4b96-b63c-93f9a7399e5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:19.169071 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbb7e40-2525-4b96-b63c-93f9a7399e5a admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:19.181228 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1021/2035] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:44:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:44:19.185652 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c3dfaf-b490-4cfe-b0e5-e0f31dc8c681 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:19.197998 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c3dfaf-b490-4cfe-b0e5-e0f31dc8c681 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:19.202670 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c44fb5-5342-43ec-b6a3-397e5e8157a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:19.203575 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c44fb5-5342-43ec-b6a3-397e5e8157a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:19.203867 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c44fb5-5342-43ec-b6a3-397e5e8157a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:19.204143 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c44fb5-5342-43ec-b6a3-397e5e8157a1 admin admin] PATH_INFO: `/v3/projects/3cee796edf9b42538358459363ea24ca/users/e9c52d463394487d94e55aed8c44068a/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:19.226129 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c44fb5-5342-43ec-b6a3-397e5e8157a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cee796edf9b42538358459363ea24ca, user_id=e9c52d463394487d94e55aed8c44068a, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:19.229309 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c44fb5-5342-43ec-b6a3-397e5e8157a1 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:19.245270 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1015/2036] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:44:19 2022] PUT /identity/v3/projects/3cee796edf9b42538358459363ea24ca/users/e9c52d463394487d94e55aed8c44068a/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:19.250718 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ac435-9112-4261-8e13-e7fae3959fc4 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:19.251100 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ac435-9112-4261-8e13-e7fae3959fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:19.251461 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ac435-9112-4261-8e13-e7fae3959fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:19.276644 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-f09ac435-9112-4261-8e13-e7fae3959fc4 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:19.277922 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-f09ac435-9112-4261-8e13-e7fae3959fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:19.345045 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1022/2037] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:44:19.354807 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c44fb5-5342-43ec-b6a3-397e5e8157a1 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:19.356566 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-92c44fb5-5342-43ec-b6a3-397e5e8157a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:19.439018 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:19.465646 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c44fb5-5342-43ec-b6a3-397e5e8157a1 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:19.470059 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b8deb-3977-4bb2-84d5-267644e758f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:19.470779 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b8deb-3977-4bb2-84d5-267644e758f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:19.471137 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b8deb-3977-4bb2-84d5-267644e758f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:19.471480 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b8deb-3977-4bb2-84d5-267644e758f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:19.481253 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b8deb-3977-4bb2-84d5-267644e758f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:19.482693 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b8deb-3977-4bb2-84d5-267644e758f4 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:19.494779 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1016/2038] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:44:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:44:20.457297 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1ad84-5f7f-4aac-8a74-b94ebcabe3d6 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:20.457804 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1ad84-5f7f-4aac-8a74-b94ebcabe3d6 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:20.458010 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1ad84-5f7f-4aac-8a74-b94ebcabe3d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:20.490320 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-b9d1ad84-5f7f-4aac-8a74-b94ebcabe3d6 None None] MFA Rules not processed for user `e9c52d463394487d94e55aed8c44068a`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:20.491251 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b9d1ad84-5f7f-4aac-8a74-b94ebcabe3d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:20.602644 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1023/2039] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:44:20.614543 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b8deb-3977-4bb2-84d5-267644e758f4 service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:20.616278 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-450b8deb-3977-4bb2-84d5-267644e758f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:20.678066 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b8deb-3977-4bb2-84d5-267644e758f4 service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:20.681642 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337cc64-9411-4b49-8753-393642281313 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:20.682286 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337cc64-9411-4b49-8753-393642281313 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:20.682569 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337cc64-9411-4b49-8753-393642281313 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:20.682845 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337cc64-9411-4b49-8753-393642281313 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:20.691580 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9337cc64-9411-4b49-8753-393642281313 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:20.693235 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9337cc64-9411-4b49-8753-393642281313 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:20.703612 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1017/2040] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:44:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:44:27.323611 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456655eb-3c3c-498f-a70a-98e63731967b None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:27.324188 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456655eb-3c3c-498f-a70a-98e63731967b None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:27.324282 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456655eb-3c3c-498f-a70a-98e63731967b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:27.345455 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-456655eb-3c3c-498f-a70a-98e63731967b None None] MFA Rules not processed for user `e9c52d463394487d94e55aed8c44068a`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:27.347052 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-456655eb-3c3c-498f-a70a-98e63731967b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:27.354986 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1024/2041] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:44:27.427501 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337cc64-9411-4b49-8753-393642281313 service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:27.428940 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-9337cc64-9411-4b49-8753-393642281313 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:27.464572 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337cc64-9411-4b49-8753-393642281313 service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:27.469222 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150c88f-ecb2-46d9-9fa6-829be95c671a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:27.470099 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150c88f-ecb2-46d9-9fa6-829be95c671a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:27.470379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150c88f-ecb2-46d9-9fa6-829be95c671a admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:27.470656 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150c88f-ecb2-46d9-9fa6-829be95c671a admin admin] PATH_INFO: `/v3/users/e607d36d04fa42439226e33517322783` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:27.471875 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4150c88f-ecb2-46d9-9fa6-829be95c671a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e607d36d04fa42439226e33517322783)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:27.474061 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4150c88f-ecb2-46d9-9fa6-829be95c671a admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:27.525561 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-4150c88f-ecb2-46d9-9fa6-829be95c671a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e607d36d04fa42439226e33517322783'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:27.526157 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-4150c88f-ecb2-46d9-9fa6-829be95c671a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e607d36d04fa42439226e33517322783'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:27.533997 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-4150c88f-ecb2-46d9-9fa6-829be95c671a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e607d36d04fa42439226e33517322783'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:27.546961 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-4150c88f-ecb2-46d9-9fa6-829be95c671a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e607d36d04fa42439226e33517322783'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:27.547847 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-4150c88f-ecb2-46d9-9fa6-829be95c671a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e607d36d04fa42439226e33517322783'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:27.564938 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1018/2042] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:44:27 2022] DELETE /identity/v3/users/e607d36d04fa42439226e33517322783 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:27.566262 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e472a28-648e-4854-93bd-4a6dbc0bbcb8 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:27.566553 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e472a28-648e-4854-93bd-4a6dbc0bbcb8 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:27.566834 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e472a28-648e-4854-93bd-4a6dbc0bbcb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:27.585493 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-0e472a28-648e-4854-93bd-4a6dbc0bbcb8 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:27.586707 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-0e472a28-648e-4854-93bd-4a6dbc0bbcb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:27.648415 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1025/2043] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:44:27.659139 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150c88f-ecb2-46d9-9fa6-829be95c671a admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:27.660753 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-4150c88f-ecb2-46d9-9fa6-829be95c671a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:27.717994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:27.744321 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150c88f-ecb2-46d9-9fa6-829be95c671a admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:27.754993 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7b61ed-35d6-481a-a160-311349436daf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:27.755517 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b61ed-35d6-481a-a160-311349436daf service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:27.756174 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b61ed-35d6-481a-a160-311349436daf service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:27.756296 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b61ed-35d6-481a-a160-311349436daf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:27.767179 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7b61ed-35d6-481a-a160-311349436daf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:27.768804 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7b61ed-35d6-481a-a160-311349436daf service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:27.780847 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1019/2044] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:44:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:44:28.191240 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e472a28-648e-4854-93bd-4a6dbc0bbcb8 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:28.193233 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-0e472a28-648e-4854-93bd-4a6dbc0bbcb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:28.206084 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e472a28-648e-4854-93bd-4a6dbc0bbcb8 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:28.211527 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae3e1fc-8722-4090-9bb8-186ccc6cb11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:28.212624 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3e1fc-8722-4090-9bb8-186ccc6cb11d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:28.212951 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3e1fc-8722-4090-9bb8-186ccc6cb11d admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:28.213296 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3e1fc-8722-4090-9bb8-186ccc6cb11d admin admin] PATH_INFO: `/v3/projects/264dc81c40134504a960161f02d76df8` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:28.214715 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae3e1fc-8722-4090-9bb8-186ccc6cb11d admin admin] RBAC: Authorizing `identity:delete_project(project_id=264dc81c40134504a960161f02d76df8)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:28.216753 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae3e1fc-8722-4090-9bb8-186ccc6cb11d admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:28.221638 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:28.283460 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-3ae3e1fc-8722-4090-9bb8-186ccc6cb11d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '264dc81c40134504a960161f02d76df8', 'request_id': 'req-3ae3e1fc-8722-4090-9bb8-186ccc6cb11d'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:28.298425 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-3ae3e1fc-8722-4090-9bb8-186ccc6cb11d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '264dc81c40134504a960161f02d76df8', 'request_id': 'req-3ae3e1fc-8722-4090-9bb8-186ccc6cb11d'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:28.304980 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-3ae3e1fc-8722-4090-9bb8-186ccc6cb11d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '264dc81c40134504a960161f02d76df8', 'request_id': 'req-3ae3e1fc-8722-4090-9bb8-186ccc6cb11d'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:28.318517 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-3ae3e1fc-8722-4090-9bb8-186ccc6cb11d admin admin] The token cache is being invalidate because project 264dc81c40134504a960161f02d76df8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:44:28.318795 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-3ae3e1fc-8722-4090-9bb8-186ccc6cb11d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:28.319450 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-3ae3e1fc-8722-4090-9bb8-186ccc6cb11d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:28.321269 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1026/2045] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:44:28 2022] DELETE /identity/v3/projects/264dc81c40134504a960161f02d76df8 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:28.325419 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7b61ed-35d6-481a-a160-311349436daf service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:28.335795 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7b61ed-35d6-481a-a160-311349436daf service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:28.401659 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2946053-0d0a-4ef8-964d-f5095c0706da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:28.402535 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2946053-0d0a-4ef8-964d-f5095c0706da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:28.402823 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2946053-0d0a-4ef8-964d-f5095c0706da admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:28.403102 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2946053-0d0a-4ef8-964d-f5095c0706da admin admin] PATH_INFO: `/v3/users/f2e35eaaf9e045a79bb28bae2f314bff` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:28.404300 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2946053-0d0a-4ef8-964d-f5095c0706da admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2e35eaaf9e045a79bb28bae2f314bff)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:28.406805 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2946053-0d0a-4ef8-964d-f5095c0706da admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:28.469702 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b2946053-0d0a-4ef8-964d-f5095c0706da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2e35eaaf9e045a79bb28bae2f314bff'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:28.470436 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b2946053-0d0a-4ef8-964d-f5095c0706da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2e35eaaf9e045a79bb28bae2f314bff'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:28.481758 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b2946053-0d0a-4ef8-964d-f5095c0706da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2e35eaaf9e045a79bb28bae2f314bff'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:28.498582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b2946053-0d0a-4ef8-964d-f5095c0706da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2e35eaaf9e045a79bb28bae2f314bff'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:28.499282 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b2946053-0d0a-4ef8-964d-f5095c0706da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2e35eaaf9e045a79bb28bae2f314bff'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:28.511706 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1020/2046] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:44:28 2022] DELETE /identity/v3/users/f2e35eaaf9e045a79bb28bae2f314bff => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:28.795321 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae3e1fc-8722-4090-9bb8-186ccc6cb11d admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:28.797466 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-3ae3e1fc-8722-4090-9bb8-186ccc6cb11d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:28.813801 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae3e1fc-8722-4090-9bb8-186ccc6cb11d admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:28.872872 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a029631-7803-406a-a8f5-730229dba7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:28.874153 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a029631-7803-406a-a8f5-730229dba7d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:28.874447 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a029631-7803-406a-a8f5-730229dba7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:28.874761 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a029631-7803-406a-a8f5-730229dba7d8 admin admin] PATH_INFO: `/v3/projects/0753ff14582b4b3e832c29c0459d08dd` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:28.876011 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a029631-7803-406a-a8f5-730229dba7d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0753ff14582b4b3e832c29c0459d08dd)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:28.878000 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a029631-7803-406a-a8f5-730229dba7d8 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:28.939433 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-9a029631-7803-406a-a8f5-730229dba7d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0753ff14582b4b3e832c29c0459d08dd', 'request_id': 'req-9a029631-7803-406a-a8f5-730229dba7d8'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:28.950508 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-9a029631-7803-406a-a8f5-730229dba7d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0753ff14582b4b3e832c29c0459d08dd', 'request_id': 'req-9a029631-7803-406a-a8f5-730229dba7d8'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:28.958122 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-9a029631-7803-406a-a8f5-730229dba7d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0753ff14582b4b3e832c29c0459d08dd', 'request_id': 'req-9a029631-7803-406a-a8f5-730229dba7d8'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:28.973364 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-9a029631-7803-406a-a8f5-730229dba7d8 admin admin] The token cache is being invalidate because project 0753ff14582b4b3e832c29c0459d08dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:44:28.973754 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-9a029631-7803-406a-a8f5-730229dba7d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:28.979981 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-9a029631-7803-406a-a8f5-730229dba7d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:28.989244 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1027/2047] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:44:28 2022] DELETE /identity/v3/projects/0753ff14582b4b3e832c29c0459d08dd => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:28.993206 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2946053-0d0a-4ef8-964d-f5095c0706da admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:28.995282 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-b2946053-0d0a-4ef8-964d-f5095c0706da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:29.008636 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2946053-0d0a-4ef8-964d-f5095c0706da admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:29.063348 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:29.071359 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c281256a-4166-4fe6-8bd0-fd1511cf952f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:29.072291 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c281256a-4166-4fe6-8bd0-fd1511cf952f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:29.072574 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c281256a-4166-4fe6-8bd0-fd1511cf952f admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:29.072883 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c281256a-4166-4fe6-8bd0-fd1511cf952f admin admin] PATH_INFO: `/v3/users/b637fc9d18df41a1aafbcea458c8fc30` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:29.074567 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c281256a-4166-4fe6-8bd0-fd1511cf952f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b637fc9d18df41a1aafbcea458c8fc30)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:29.077831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c281256a-4166-4fe6-8bd0-fd1511cf952f admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:29.133538 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-c281256a-4166-4fe6-8bd0-fd1511cf952f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b637fc9d18df41a1aafbcea458c8fc30'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:29.134182 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-c281256a-4166-4fe6-8bd0-fd1511cf952f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b637fc9d18df41a1aafbcea458c8fc30'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:29.141753 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-c281256a-4166-4fe6-8bd0-fd1511cf952f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b637fc9d18df41a1aafbcea458c8fc30'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:29.154061 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-c281256a-4166-4fe6-8bd0-fd1511cf952f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b637fc9d18df41a1aafbcea458c8fc30'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:29.154653 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-c281256a-4166-4fe6-8bd0-fd1511cf952f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b637fc9d18df41a1aafbcea458c8fc30'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:29.168151 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1021/2048] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:44:28 2022] DELETE /identity/v3/users/b637fc9d18df41a1aafbcea458c8fc30 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:29.453982 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a029631-7803-406a-a8f5-730229dba7d8 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:29.455535 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-9a029631-7803-406a-a8f5-730229dba7d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:29.470957 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a029631-7803-406a-a8f5-730229dba7d8 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:29.489762 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:29.530001 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4e434d-7a38-456d-93bd-365ba45dbaed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:29.530865 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e434d-7a38-456d-93bd-365ba45dbaed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:29.531762 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e434d-7a38-456d-93bd-365ba45dbaed admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:29.532048 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e434d-7a38-456d-93bd-365ba45dbaed admin admin] PATH_INFO: `/v3/projects/3f09461186e94571a5e5cd3c43471f51` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:29.533347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4e434d-7a38-456d-93bd-365ba45dbaed admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f09461186e94571a5e5cd3c43471f51)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:29.535292 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4e434d-7a38-456d-93bd-365ba45dbaed admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:29.584680 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-1b4e434d-7a38-456d-93bd-365ba45dbaed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f09461186e94571a5e5cd3c43471f51', 'request_id': 'req-1b4e434d-7a38-456d-93bd-365ba45dbaed'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:29.590068 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-1b4e434d-7a38-456d-93bd-365ba45dbaed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f09461186e94571a5e5cd3c43471f51', 'request_id': 'req-1b4e434d-7a38-456d-93bd-365ba45dbaed'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:29.596306 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-1b4e434d-7a38-456d-93bd-365ba45dbaed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f09461186e94571a5e5cd3c43471f51', 'request_id': 'req-1b4e434d-7a38-456d-93bd-365ba45dbaed'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:29.608445 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-1b4e434d-7a38-456d-93bd-365ba45dbaed admin admin] The token cache is being invalidate because project 3f09461186e94571a5e5cd3c43471f51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:44:29.608734 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-1b4e434d-7a38-456d-93bd-365ba45dbaed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:29.609370 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-1b4e434d-7a38-456d-93bd-365ba45dbaed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:29.610922 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1028/2049] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:44:29 2022] DELETE /identity/v3/projects/3f09461186e94571a5e5cd3c43471f51 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:29.615947 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ba21f-4faf-4c6c-941d-9a1f7d05ba81 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:29.616535 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ba21f-4faf-4c6c-941d-9a1f7d05ba81 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:29.616535 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ba21f-4faf-4c6c-941d-9a1f7d05ba81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:29.645141 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-045ba21f-4faf-4c6c-941d-9a1f7d05ba81 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:29.646304 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-045ba21f-4faf-4c6c-941d-9a1f7d05ba81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:29.707193 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1022/2050] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:44:29.716133 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e7329-4c7e-4622-b62d-9d5f5010b6f8 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:29.716431 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e7329-4c7e-4622-b62d-9d5f5010b6f8 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:29.716716 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e7329-4c7e-4622-b62d-9d5f5010b6f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:29.736133 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-fe7e7329-4c7e-4622-b62d-9d5f5010b6f8 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:29.737369 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-fe7e7329-4c7e-4622-b62d-9d5f5010b6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:29.745180 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1029/2051] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:44:29.749436 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045ba21f-4faf-4c6c-941d-9a1f7d05ba81 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:29.760156 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045ba21f-4faf-4c6c-941d-9a1f7d05ba81 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:29.764871 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb41ba7b-8ca1-4c54-a548-0e2b3aa5a65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:29.766194 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41ba7b-8ca1-4c54-a548-0e2b3aa5a65f admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:29.766483 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41ba7b-8ca1-4c54-a548-0e2b3aa5a65f admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:29.766764 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41ba7b-8ca1-4c54-a548-0e2b3aa5a65f admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:29.767506 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb41ba7b-8ca1-4c54-a548-0e2b3aa5a65f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:44:29.767820 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb41ba7b-8ca1-4c54-a548-0e2b3aa5a65f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:29.769491 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb41ba7b-8ca1-4c54-a548-0e2b3aa5a65f admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:29.781554 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1023/2052] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:44:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:44:29.786136 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7e7329-4c7e-4622-b62d-9d5f5010b6f8 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:29.796941 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7e7329-4c7e-4622-b62d-9d5f5010b6f8 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:29.802755 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3edd860-d228-4270-aad4-c82b0d8ffe6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:29.803626 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3edd860-d228-4270-aad4-c82b0d8ffe6a admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:29.804067 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3edd860-d228-4270-aad4-c82b0d8ffe6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:29.804351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3edd860-d228-4270-aad4-c82b0d8ffe6a admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:29.805204 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3edd860-d228-4270-aad4-c82b0d8ffe6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:29.807226 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3edd860-d228-4270-aad4-c82b0d8ffe6a admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:29.833774 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1030/2053] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:44:29 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:44:29.837982 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb41ba7b-8ca1-4c54-a548-0e2b3aa5a65f admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:29.847998 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb41ba7b-8ca1-4c54-a548-0e2b3aa5a65f admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:29.899260 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:29.907288 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b391efcb-6f86-4ae5-8c56-87a20742e78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:29.908195 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b391efcb-6f86-4ae5-8c56-87a20742e78d admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:29.908730 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b391efcb-6f86-4ae5-8c56-87a20742e78d admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:29.909053 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b391efcb-6f86-4ae5-8c56-87a20742e78d admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:29.910001 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b391efcb-6f86-4ae5-8c56-87a20742e78d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:29.911984 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b391efcb-6f86-4ae5-8c56-87a20742e78d admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:29.941330 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1024/2054] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:44:29 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:44:29.942822 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3edd860-d228-4270-aad4-c82b0d8ffe6a admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:29.953091 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3edd860-d228-4270-aad4-c82b0d8ffe6a admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:29.958300 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e9b5e-b0c1-48d8-97f6-51f43cc6e1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:29.959012 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e9b5e-b0c1-48d8-97f6-51f43cc6e1a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:29.959387 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e9b5e-b0c1-48d8-97f6-51f43cc6e1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:29.959764 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e9b5e-b0c1-48d8-97f6-51f43cc6e1a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:29.960560 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e9b5e-b0c1-48d8-97f6-51f43cc6e1a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:29.962270 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e9b5e-b0c1-48d8-97f6-51f43cc6e1a7 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:29.973588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1031/2055] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:44:29.978044 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b391efcb-6f86-4ae5-8c56-87a20742e78d admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:29.991886 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b391efcb-6f86-4ae5-8c56-87a20742e78d admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:29.998549 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e560398-f33d-4fc2-951c-328f45020aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:29.999233 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e560398-f33d-4fc2-951c-328f45020aeb admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:29.999529 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e560398-f33d-4fc2-951c-328f45020aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:29.999867 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e560398-f33d-4fc2-951c-328f45020aeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:30.001132 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e560398-f33d-4fc2-951c-328f45020aeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:30.002796 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e560398-f33d-4fc2-951c-328f45020aeb admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:30.015411 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1025/2056] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:44:30.019594 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e9b5e-b0c1-48d8-97f6-51f43cc6e1a7 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:30.030464 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e9b5e-b0c1-48d8-97f6-51f43cc6e1a7 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:30.046049 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86dc885-2dd3-47ad-b089-56869c308f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:30.047680 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86dc885-2dd3-47ad-b089-56869c308f1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:30.048089 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86dc885-2dd3-47ad-b089-56869c308f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:30.048440 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86dc885-2dd3-47ad-b089-56869c308f1b admin admin] PATH_INFO: `/v3/projects/f9f610de4ec9488bb18cb6035c6ddf13/users/2cc959031b874096b4ff50e388143600/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:30.069885 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86dc885-2dd3-47ad-b089-56869c308f1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9f610de4ec9488bb18cb6035c6ddf13, user_id=2cc959031b874096b4ff50e388143600, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:30.072575 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86dc885-2dd3-47ad-b089-56869c308f1b admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:30.086415 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1032/2057] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:44:30 2022] PUT /identity/v3/projects/f9f610de4ec9488bb18cb6035c6ddf13/users/2cc959031b874096b4ff50e388143600/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:30.099854 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b6d214-a9ad-422c-b8e1-a6e0f9e42e0a None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:30.100247 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b6d214-a9ad-422c-b8e1-a6e0f9e42e0a None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:30.100603 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b6d214-a9ad-422c-b8e1-a6e0f9e42e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:30.129372 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-06b6d214-a9ad-422c-b8e1-a6e0f9e42e0a None None] MFA Rules not processed for user `2cc959031b874096b4ff50e388143600`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:30.131472 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-06b6d214-a9ad-422c-b8e1-a6e0f9e42e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:30.212788 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1026/2058] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:44:30.220870 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86dc885-2dd3-47ad-b089-56869c308f1b admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:30.222999 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e86dc885-2dd3-47ad-b089-56869c308f1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:30.276205 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:30.280517 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86dc885-2dd3-47ad-b089-56869c308f1b admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:30.286486 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd9cf81-ad87-46e3-9bc3-2d21fa047902 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:30.287399 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd9cf81-ad87-46e3-9bc3-2d21fa047902 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:30.287897 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd9cf81-ad87-46e3-9bc3-2d21fa047902 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:30.288420 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd9cf81-ad87-46e3-9bc3-2d21fa047902 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:30.298163 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd9cf81-ad87-46e3-9bc3-2d21fa047902 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:30.299616 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd9cf81-ad87-46e3-9bc3-2d21fa047902 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:30.314121 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1033/2059] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:44:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:44:36.054021 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b6d214-a9ad-422c-b8e1-a6e0f9e42e0a None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:36.127526 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b6d214-a9ad-422c-b8e1-a6e0f9e42e0a None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:36.132384 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc635983-93ed-4c81-b1ad-6d9e771ad1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:36.133261 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc635983-93ed-4c81-b1ad-6d9e771ad1a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:36.133545 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc635983-93ed-4c81-b1ad-6d9e771ad1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:36.133824 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc635983-93ed-4c81-b1ad-6d9e771ad1a6 admin admin] PATH_INFO: `/v3/users/2cc959031b874096b4ff50e388143600` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:36.135107 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc635983-93ed-4c81-b1ad-6d9e771ad1a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cc959031b874096b4ff50e388143600)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:36.137038 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc635983-93ed-4c81-b1ad-6d9e771ad1a6 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:36.191363 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-bc635983-93ed-4c81-b1ad-6d9e771ad1a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cc959031b874096b4ff50e388143600'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:36.191717 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-bc635983-93ed-4c81-b1ad-6d9e771ad1a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cc959031b874096b4ff50e388143600'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:36.199464 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-bc635983-93ed-4c81-b1ad-6d9e771ad1a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cc959031b874096b4ff50e388143600'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:36.207727 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:36.212821 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-bc635983-93ed-4c81-b1ad-6d9e771ad1a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cc959031b874096b4ff50e388143600'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:36.213580 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-bc635983-93ed-4c81-b1ad-6d9e771ad1a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cc959031b874096b4ff50e388143600'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:36.231822 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1027/2060] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:44:36 2022] DELETE /identity/v3/users/2cc959031b874096b4ff50e388143600 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:44:36.236761 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ed6be-0fa4-44a7-85bb-b25ee48a22e9 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:36.237295 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ed6be-0fa4-44a7-85bb-b25ee48a22e9 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:36.237394 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ed6be-0fa4-44a7-85bb-b25ee48a22e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:36.255767 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-693ed6be-0fa4-44a7-85bb-b25ee48a22e9 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:36.257245 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-693ed6be-0fa4-44a7-85bb-b25ee48a22e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:36.327434 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1034/2061] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:44:36.338869 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc635983-93ed-4c81-b1ad-6d9e771ad1a6 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:36.340342 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-bc635983-93ed-4c81-b1ad-6d9e771ad1a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:36.397984 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc635983-93ed-4c81-b1ad-6d9e771ad1a6 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:36.401587 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b83c96a-4997-4ab5-81e7-337ac2579adb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:36.402234 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83c96a-4997-4ab5-81e7-337ac2579adb service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:36.402522 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83c96a-4997-4ab5-81e7-337ac2579adb service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:36.402800 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83c96a-4997-4ab5-81e7-337ac2579adb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:36.412320 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b83c96a-4997-4ab5-81e7-337ac2579adb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:36.413794 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b83c96a-4997-4ab5-81e7-337ac2579adb service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:36.428035 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1028/2062] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:44:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:44:36.706301 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ed6be-0fa4-44a7-85bb-b25ee48a22e9 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:36.708398 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-693ed6be-0fa4-44a7-85bb-b25ee48a22e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:36.734681 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ed6be-0fa4-44a7-85bb-b25ee48a22e9 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:36.742436 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1352a1b4-2ac0-48eb-a325-fce2675ab502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:36.743596 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352a1b4-2ac0-48eb-a325-fce2675ab502 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:36.743943 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352a1b4-2ac0-48eb-a325-fce2675ab502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:36.744230 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352a1b4-2ac0-48eb-a325-fce2675ab502 admin admin] PATH_INFO: `/v3/projects/f9f610de4ec9488bb18cb6035c6ddf13` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:36.745741 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1352a1b4-2ac0-48eb-a325-fce2675ab502 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9f610de4ec9488bb18cb6035c6ddf13)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:36.756510 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1352a1b4-2ac0-48eb-a325-fce2675ab502 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:36.828506 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-1352a1b4-2ac0-48eb-a325-fce2675ab502 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9f610de4ec9488bb18cb6035c6ddf13', 'request_id': 'req-1352a1b4-2ac0-48eb-a325-fce2675ab502'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:36.836282 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-1352a1b4-2ac0-48eb-a325-fce2675ab502 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9f610de4ec9488bb18cb6035c6ddf13', 'request_id': 'req-1352a1b4-2ac0-48eb-a325-fce2675ab502'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:36.844074 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-1352a1b4-2ac0-48eb-a325-fce2675ab502 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9f610de4ec9488bb18cb6035c6ddf13', 'request_id': 'req-1352a1b4-2ac0-48eb-a325-fce2675ab502'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:36.867579 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-1352a1b4-2ac0-48eb-a325-fce2675ab502 admin admin] The token cache is being invalidate because project f9f610de4ec9488bb18cb6035c6ddf13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:44:36.868101 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-1352a1b4-2ac0-48eb-a325-fce2675ab502 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:36.869376 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-1352a1b4-2ac0-48eb-a325-fce2675ab502 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:36.872321 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1035/2063] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:44:36 2022] DELETE /identity/v3/projects/f9f610de4ec9488bb18cb6035c6ddf13 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:36.877437 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec5a7c-78a9-42b6-b23f-120a670e7a8f None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:36.877925 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec5a7c-78a9-42b6-b23f-120a670e7a8f None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:36.878453 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec5a7c-78a9-42b6-b23f-120a670e7a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:36.914843 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-8aec5a7c-78a9-42b6-b23f-120a670e7a8f None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:36.920576 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-8aec5a7c-78a9-42b6-b23f-120a670e7a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:37.003172 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1029/2064] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:44:37.011333 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d9959-4def-41ef-9572-cc95ea683b29 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:37.011659 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d9959-4def-41ef-9572-cc95ea683b29 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:37.011949 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d9959-4def-41ef-9572-cc95ea683b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:37.036920 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-e99d9959-4def-41ef-9572-cc95ea683b29 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:37.038218 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e99d9959-4def-41ef-9572-cc95ea683b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:37.047328 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1036/2065] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:44:37.050749 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec5a7c-78a9-42b6-b23f-120a670e7a8f None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:37.066232 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec5a7c-78a9-42b6-b23f-120a670e7a8f None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:37.071096 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769cca5c-007b-45f3-a18e-41afaba4bedd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:37.071770 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769cca5c-007b-45f3-a18e-41afaba4bedd admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:37.072054 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769cca5c-007b-45f3-a18e-41afaba4bedd admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:37.072335 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769cca5c-007b-45f3-a18e-41afaba4bedd admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:37.073077 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769cca5c-007b-45f3-a18e-41afaba4bedd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:44:37.073416 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769cca5c-007b-45f3-a18e-41afaba4bedd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:37.075586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769cca5c-007b-45f3-a18e-41afaba4bedd admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:37.088662 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1030/2066] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:44:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:44:37.094565 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99d9959-4def-41ef-9572-cc95ea683b29 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:37.105571 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99d9959-4def-41ef-9572-cc95ea683b29 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:37.111066 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e6a28-8318-4df8-8f4f-06bf6dcce25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:37.112119 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e6a28-8318-4df8-8f4f-06bf6dcce25c admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:37.112476 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e6a28-8318-4df8-8f4f-06bf6dcce25c admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:37.112997 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e6a28-8318-4df8-8f4f-06bf6dcce25c admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:37.114385 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730e6a28-8318-4df8-8f4f-06bf6dcce25c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:37.116640 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730e6a28-8318-4df8-8f4f-06bf6dcce25c admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:37.161279 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1037/2067] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:44:37 2022] POST /identity/v3/projects => generated 371 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:44:37.163869 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769cca5c-007b-45f3-a18e-41afaba4bedd admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:37.176370 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769cca5c-007b-45f3-a18e-41afaba4bedd admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:37.269348 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc5dd7f-8115-4bf8-9d10-8c2db42a2e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:37.277398 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5dd7f-8115-4bf8-9d10-8c2db42a2e60 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:37.277398 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5dd7f-8115-4bf8-9d10-8c2db42a2e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:37.277398 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5dd7f-8115-4bf8-9d10-8c2db42a2e60 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:37.277398 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc5dd7f-8115-4bf8-9d10-8c2db42a2e60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:37.277398 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc5dd7f-8115-4bf8-9d10-8c2db42a2e60 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:37.300973 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1031/2068] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:44:37 2022] POST /identity/v3/users => generated 286 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:44:37.302522 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e6a28-8318-4df8-8f4f-06bf6dcce25c admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:37.314356 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e6a28-8318-4df8-8f4f-06bf6dcce25c admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:37.325907 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e37ea-a7a4-47cb-9c98-b8280342b519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:37.326469 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e37ea-a7a4-47cb-9c98-b8280342b519 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:37.326977 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e37ea-a7a4-47cb-9c98-b8280342b519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:37.327319 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e37ea-a7a4-47cb-9c98-b8280342b519 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:37.329540 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148e37ea-a7a4-47cb-9c98-b8280342b519 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:37.331480 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148e37ea-a7a4-47cb-9c98-b8280342b519 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:37.345050 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1038/2069] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:44:37 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:44:37.349627 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc5dd7f-8115-4bf8-9d10-8c2db42a2e60 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:37.356505 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:37.364847 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc5dd7f-8115-4bf8-9d10-8c2db42a2e60 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:37.373500 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca5b3c6-1cc6-4861-a864-a88c032c12d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:37.374438 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca5b3c6-1cc6-4861-a864-a88c032c12d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:37.375558 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca5b3c6-1cc6-4861-a864-a88c032c12d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:37.375731 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca5b3c6-1cc6-4861-a864-a88c032c12d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:37.376741 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca5b3c6-1cc6-4861-a864-a88c032c12d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:37.379007 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca5b3c6-1cc6-4861-a864-a88c032c12d8 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:37.397310 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1032/2070] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:44:37 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:44:37.398999 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e37ea-a7a4-47cb-9c98-b8280342b519 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:37.410783 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e37ea-a7a4-47cb-9c98-b8280342b519 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:37.416970 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2baf012-1543-4891-9326-520eb20a4b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:37.418130 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2baf012-1543-4891-9326-520eb20a4b73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:37.418515 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2baf012-1543-4891-9326-520eb20a4b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:37.418899 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2baf012-1543-4891-9326-520eb20a4b73 admin admin] PATH_INFO: `/v3/projects/c47faadfc8334eada0ff2f7e04080b2c/users/7bf96a8e0a7e4fafa3451e229de9d694/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:37.438965 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2baf012-1543-4891-9326-520eb20a4b73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c47faadfc8334eada0ff2f7e04080b2c, user_id=7bf96a8e0a7e4fafa3451e229de9d694, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:37.450095 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2baf012-1543-4891-9326-520eb20a4b73 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:37.477099 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1039/2071] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:44:37 2022] PUT /identity/v3/projects/c47faadfc8334eada0ff2f7e04080b2c/users/7bf96a8e0a7e4fafa3451e229de9d694/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:37.484787 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ef49c-7b62-47fc-96df-2acd3e4e8c86 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:37.485232 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ef49c-7b62-47fc-96df-2acd3e4e8c86 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:37.485583 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ef49c-7b62-47fc-96df-2acd3e4e8c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:37.520028 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-dd0ef49c-7b62-47fc-96df-2acd3e4e8c86 None None] MFA Rules not processed for user `7bf96a8e0a7e4fafa3451e229de9d694`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:37.521847 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-dd0ef49c-7b62-47fc-96df-2acd3e4e8c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:37.600285 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1033/2072] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:44:37.612136 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2baf012-1543-4891-9326-520eb20a4b73 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:37.613852 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-f2baf012-1543-4891-9326-520eb20a4b73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:37.629853 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:37.681079 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2baf012-1543-4891-9326-520eb20a4b73 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:37.685670 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca58d89-3739-4412-9f58-dff628bd440e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:37.686367 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca58d89-3739-4412-9f58-dff628bd440e service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:37.686631 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca58d89-3739-4412-9f58-dff628bd440e service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:37.687706 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca58d89-3739-4412-9f58-dff628bd440e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:37.696775 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca58d89-3739-4412-9f58-dff628bd440e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:37.698512 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca58d89-3739-4412-9f58-dff628bd440e service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:37.708377 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1040/2073] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:44:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:44:37.736762 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ef49c-7b62-47fc-96df-2acd3e4e8c86 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:37.816771 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ef49c-7b62-47fc-96df-2acd3e4e8c86 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:37.821517 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2889ac-4238-4e4c-942f-62cf53e1bf83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:37.822394 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2889ac-4238-4e4c-942f-62cf53e1bf83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:37.822677 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2889ac-4238-4e4c-942f-62cf53e1bf83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:37.822950 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2889ac-4238-4e4c-942f-62cf53e1bf83 admin admin] PATH_INFO: `/v3/users/7bf96a8e0a7e4fafa3451e229de9d694` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:37.824166 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2889ac-4238-4e4c-942f-62cf53e1bf83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bf96a8e0a7e4fafa3451e229de9d694)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:37.826209 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2889ac-4238-4e4c-942f-62cf53e1bf83 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:37.879555 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8e2889ac-4238-4e4c-942f-62cf53e1bf83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bf96a8e0a7e4fafa3451e229de9d694'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:37.880554 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8e2889ac-4238-4e4c-942f-62cf53e1bf83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bf96a8e0a7e4fafa3451e229de9d694'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:37.893030 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8e2889ac-4238-4e4c-942f-62cf53e1bf83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bf96a8e0a7e4fafa3451e229de9d694'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:37.906413 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8e2889ac-4238-4e4c-942f-62cf53e1bf83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bf96a8e0a7e4fafa3451e229de9d694'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:37.907156 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8e2889ac-4238-4e4c-942f-62cf53e1bf83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bf96a8e0a7e4fafa3451e229de9d694'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:37.918077 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:37.928243 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1034/2074] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:44:37 2022] DELETE /identity/v3/users/7bf96a8e0a7e4fafa3451e229de9d694 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:37.931055 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d79f61-68f9-41df-aa40-2d04274172f1 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:37.931502 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d79f61-68f9-41df-aa40-2d04274172f1 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:37.931828 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d79f61-68f9-41df-aa40-2d04274172f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:37.973066 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-07d79f61-68f9-41df-aa40-2d04274172f1 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:37.978352 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-07d79f61-68f9-41df-aa40-2d04274172f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:38.060674 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1041/2075] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:44:38.085878 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2889ac-4238-4e4c-942f-62cf53e1bf83 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:38.088422 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-8e2889ac-4238-4e4c-942f-62cf53e1bf83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:38.161480 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2889ac-4238-4e4c-942f-62cf53e1bf83 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:38.165059 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb59695-4469-48a8-971a-0d6d37ec47c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:38.165743 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb59695-4469-48a8-971a-0d6d37ec47c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:38.166031 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb59695-4469-48a8-971a-0d6d37ec47c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:38.166313 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb59695-4469-48a8-971a-0d6d37ec47c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:38.176136 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb59695-4469-48a8-971a-0d6d37ec47c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:38.177673 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb59695-4469-48a8-971a-0d6d37ec47c5 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:38.188171 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1035/2076] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:44:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:44:38.306283 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d79f61-68f9-41df-aa40-2d04274172f1 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:38.307921 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-07d79f61-68f9-41df-aa40-2d04274172f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:38.320681 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d79f61-68f9-41df-aa40-2d04274172f1 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:38.329239 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bb974d-cf56-4186-909e-8b61065d09f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:38.330284 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb974d-cf56-4186-909e-8b61065d09f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:38.330589 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb974d-cf56-4186-909e-8b61065d09f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:38.330845 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb974d-cf56-4186-909e-8b61065d09f6 admin admin] PATH_INFO: `/v3/users/3369ab1f03f44454a344c225ec66c1af` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:38.332180 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bb974d-cf56-4186-909e-8b61065d09f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3369ab1f03f44454a344c225ec66c1af)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:38.334253 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bb974d-cf56-4186-909e-8b61065d09f6 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:38.390804 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-18bb974d-cf56-4186-909e-8b61065d09f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3369ab1f03f44454a344c225ec66c1af'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:38.391575 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-18bb974d-cf56-4186-909e-8b61065d09f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3369ab1f03f44454a344c225ec66c1af'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:38.392345 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-18bb974d-cf56-4186-909e-8b61065d09f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3369ab1f03f44454a344c225ec66c1af'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:38.401630 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-18bb974d-cf56-4186-909e-8b61065d09f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3369ab1f03f44454a344c225ec66c1af'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:38.419686 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-18bb974d-cf56-4186-909e-8b61065d09f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3369ab1f03f44454a344c225ec66c1af'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:38.444737 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1042/2077] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:44:38 2022] DELETE /identity/v3/users/3369ab1f03f44454a344c225ec66c1af => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:38.447398 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7bf5df-e871-4079-8120-65898f54cca4 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:38.447957 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7bf5df-e871-4079-8120-65898f54cca4 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:38.448346 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7bf5df-e871-4079-8120-65898f54cca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:38.474010 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-4b7bf5df-e871-4079-8120-65898f54cca4 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:38.476728 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-4b7bf5df-e871-4079-8120-65898f54cca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:38.598728 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1036/2078] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:44:38.638749 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bb974d-cf56-4186-909e-8b61065d09f6 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:38.641108 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-18bb974d-cf56-4186-909e-8b61065d09f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:38.682660 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:38.733399 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bb974d-cf56-4186-909e-8b61065d09f6 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:38.736671 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdccc0f3-f301-4723-a3c3-ee7554a0aa99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:38.737583 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdccc0f3-f301-4723-a3c3-ee7554a0aa99 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:38.737973 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdccc0f3-f301-4723-a3c3-ee7554a0aa99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:38.738325 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdccc0f3-f301-4723-a3c3-ee7554a0aa99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:38.739905 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7bf5df-e871-4079-8120-65898f54cca4 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:38.741385 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-4b7bf5df-e871-4079-8120-65898f54cca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:38.749217 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdccc0f3-f301-4723-a3c3-ee7554a0aa99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:38.753225 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdccc0f3-f301-4723-a3c3-ee7554a0aa99 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:38.754901 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7bf5df-e871-4079-8120-65898f54cca4 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:38.761543 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c974f56-247d-4813-8d57-595750d62359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:38.762587 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c974f56-247d-4813-8d57-595750d62359 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:38.762983 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c974f56-247d-4813-8d57-595750d62359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:38.763373 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c974f56-247d-4813-8d57-595750d62359 admin admin] PATH_INFO: `/v3/projects/c47faadfc8334eada0ff2f7e04080b2c` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:38.764830 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c974f56-247d-4813-8d57-595750d62359 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c47faadfc8334eada0ff2f7e04080b2c)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:38.766725 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1043/2079] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:44:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:44:38.767791 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c974f56-247d-4813-8d57-595750d62359 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:38.849103 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-7c974f56-247d-4813-8d57-595750d62359 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c47faadfc8334eada0ff2f7e04080b2c', 'request_id': 'req-7c974f56-247d-4813-8d57-595750d62359'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:38.859138 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-7c974f56-247d-4813-8d57-595750d62359 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c47faadfc8334eada0ff2f7e04080b2c', 'request_id': 'req-7c974f56-247d-4813-8d57-595750d62359'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:38.866849 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-7c974f56-247d-4813-8d57-595750d62359 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c47faadfc8334eada0ff2f7e04080b2c', 'request_id': 'req-7c974f56-247d-4813-8d57-595750d62359'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:38.884535 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-7c974f56-247d-4813-8d57-595750d62359 admin admin] The token cache is being invalidate because project c47faadfc8334eada0ff2f7e04080b2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:44:38.884977 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-7c974f56-247d-4813-8d57-595750d62359 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:38.885717 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-7c974f56-247d-4813-8d57-595750d62359 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:38.888573 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1037/2080] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:44:38 2022] DELETE /identity/v3/projects/c47faadfc8334eada0ff2f7e04080b2c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:44:39.341553 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdccc0f3-f301-4723-a3c3-ee7554a0aa99 service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:39.344014 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-cdccc0f3-f301-4723-a3c3-ee7554a0aa99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:39.357533 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdccc0f3-f301-4723-a3c3-ee7554a0aa99 service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:39.422135 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d58e66-afff-483d-9237-f7449fa1b41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:39.423009 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d58e66-afff-483d-9237-f7449fa1b41d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:39.423318 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d58e66-afff-483d-9237-f7449fa1b41d admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:39.423578 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d58e66-afff-483d-9237-f7449fa1b41d admin admin] PATH_INFO: `/v3/projects/fd67d9207fc64ec29ea1bb9cf008c0ec` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:39.424767 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d58e66-afff-483d-9237-f7449fa1b41d admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd67d9207fc64ec29ea1bb9cf008c0ec)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:39.426727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d58e66-afff-483d-9237-f7449fa1b41d admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:39.488954 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-73d58e66-afff-483d-9237-f7449fa1b41d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd67d9207fc64ec29ea1bb9cf008c0ec', 'request_id': 'req-73d58e66-afff-483d-9237-f7449fa1b41d'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:39.495700 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-73d58e66-afff-483d-9237-f7449fa1b41d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd67d9207fc64ec29ea1bb9cf008c0ec', 'request_id': 'req-73d58e66-afff-483d-9237-f7449fa1b41d'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:39.510156 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-73d58e66-afff-483d-9237-f7449fa1b41d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd67d9207fc64ec29ea1bb9cf008c0ec', 'request_id': 'req-73d58e66-afff-483d-9237-f7449fa1b41d'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:39.559430 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-73d58e66-afff-483d-9237-f7449fa1b41d admin admin] The token cache is being invalidate because project fd67d9207fc64ec29ea1bb9cf008c0ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:44:39.559732 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-73d58e66-afff-483d-9237-f7449fa1b41d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:39.561057 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-73d58e66-afff-483d-9237-f7449fa1b41d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:39.563552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1044/2081] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:44:39 2022] DELETE /identity/v3/projects/fd67d9207fc64ec29ea1bb9cf008c0ec => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:39.568365 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c974f56-247d-4813-8d57-595750d62359 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:39.572058 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-7c974f56-247d-4813-8d57-595750d62359 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:39.598939 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c974f56-247d-4813-8d57-595750d62359 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:39.636159 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:39.679179 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314de25b-6c28-423a-8cd7-caea04f33989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:39.680084 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314de25b-6c28-423a-8cd7-caea04f33989 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:39.680968 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314de25b-6c28-423a-8cd7-caea04f33989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:39.680968 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314de25b-6c28-423a-8cd7-caea04f33989 admin admin] PATH_INFO: `/v3/users/018effefcef14d9ca86e200666cbc2e5` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:39.682551 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314de25b-6c28-423a-8cd7-caea04f33989 admin admin] RBAC: Authorizing `identity:delete_user(user_id=018effefcef14d9ca86e200666cbc2e5)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:39.684503 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314de25b-6c28-423a-8cd7-caea04f33989 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:39.756531 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-314de25b-6c28-423a-8cd7-caea04f33989 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '018effefcef14d9ca86e200666cbc2e5'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:39.757492 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-314de25b-6c28-423a-8cd7-caea04f33989 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '018effefcef14d9ca86e200666cbc2e5'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:39.766636 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-314de25b-6c28-423a-8cd7-caea04f33989 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '018effefcef14d9ca86e200666cbc2e5'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:39.781643 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-314de25b-6c28-423a-8cd7-caea04f33989 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '018effefcef14d9ca86e200666cbc2e5'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:39.782377 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-314de25b-6c28-423a-8cd7-caea04f33989 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '018effefcef14d9ca86e200666cbc2e5'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:39.796229 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1038/2082] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:44:39 2022] DELETE /identity/v3/users/018effefcef14d9ca86e200666cbc2e5 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:40.225041 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d58e66-afff-483d-9237-f7449fa1b41d admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:40.227249 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-73d58e66-afff-483d-9237-f7449fa1b41d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:40.240496 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d58e66-afff-483d-9237-f7449fa1b41d admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:40.303387 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46f7a2-8b2b-468d-8490-c93747394c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:40.304182 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46f7a2-8b2b-468d-8490-c93747394c8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:40.304466 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46f7a2-8b2b-468d-8490-c93747394c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:40.304798 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46f7a2-8b2b-468d-8490-c93747394c8e admin admin] PATH_INFO: `/v3/projects/c53d547411554dd69679ad3935bb7510` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:40.305967 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f46f7a2-8b2b-468d-8490-c93747394c8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c53d547411554dd69679ad3935bb7510)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:40.307913 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f46f7a2-8b2b-468d-8490-c93747394c8e admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:40.309498 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:40.360669 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6f46f7a2-8b2b-468d-8490-c93747394c8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c53d547411554dd69679ad3935bb7510', 'request_id': 'req-6f46f7a2-8b2b-468d-8490-c93747394c8e'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:40.366179 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6f46f7a2-8b2b-468d-8490-c93747394c8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c53d547411554dd69679ad3935bb7510', 'request_id': 'req-6f46f7a2-8b2b-468d-8490-c93747394c8e'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:40.372306 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6f46f7a2-8b2b-468d-8490-c93747394c8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c53d547411554dd69679ad3935bb7510', 'request_id': 'req-6f46f7a2-8b2b-468d-8490-c93747394c8e'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:40.386467 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6f46f7a2-8b2b-468d-8490-c93747394c8e admin admin] The token cache is being invalidate because project c53d547411554dd69679ad3935bb7510 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:44:40.386756 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6f46f7a2-8b2b-468d-8490-c93747394c8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:40.387429 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6f46f7a2-8b2b-468d-8490-c93747394c8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:40.389050 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1045/2083] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:44:40 2022] DELETE /identity/v3/projects/c53d547411554dd69679ad3935bb7510 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:40.394680 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be742694-3af4-4f07-a8f5-20c3064bd3f8 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:40.395133 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be742694-3af4-4f07-a8f5-20c3064bd3f8 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:40.395481 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be742694-3af4-4f07-a8f5-20c3064bd3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:40.425936 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-be742694-3af4-4f07-a8f5-20c3064bd3f8 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:40.427278 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-be742694-3af4-4f07-a8f5-20c3064bd3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:40.509791 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1039/2084] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:44:40.525396 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdfb717-417b-4013-b25f-c274882048de None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:40.525848 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdfb717-417b-4013-b25f-c274882048de None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:40.526306 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdfb717-417b-4013-b25f-c274882048de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:40.552194 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-cbdfb717-417b-4013-b25f-c274882048de None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:40.554005 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-cbdfb717-417b-4013-b25f-c274882048de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:40.565211 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1046/2085] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:44:40.570168 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be742694-3af4-4f07-a8f5-20c3064bd3f8 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:40.582121 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be742694-3af4-4f07-a8f5-20c3064bd3f8 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:40.587283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518cf157-d35d-42d1-86d6-e92e7bab2339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:40.588135 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518cf157-d35d-42d1-86d6-e92e7bab2339 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:40.588430 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518cf157-d35d-42d1-86d6-e92e7bab2339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:40.588802 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518cf157-d35d-42d1-86d6-e92e7bab2339 admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:40.589642 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518cf157-d35d-42d1-86d6-e92e7bab2339 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:44:40.590046 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518cf157-d35d-42d1-86d6-e92e7bab2339 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:40.591600 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518cf157-d35d-42d1-86d6-e92e7bab2339 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:40.605054 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1040/2086] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:44:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:44:40.613714 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdfb717-417b-4013-b25f-c274882048de None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:40.643194 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdfb717-417b-4013-b25f-c274882048de None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:40.649585 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c9c99-e959-4ed8-9bd7-aaf53fb51d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:40.650516 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c9c99-e959-4ed8-9bd7-aaf53fb51d0c admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:40.650950 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c9c99-e959-4ed8-9bd7-aaf53fb51d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:40.651373 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c9c99-e959-4ed8-9bd7-aaf53fb51d0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:40.652219 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3c9c99-e959-4ed8-9bd7-aaf53fb51d0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:40.654367 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3c9c99-e959-4ed8-9bd7-aaf53fb51d0c admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:40.697144 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1047/2087] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:44:40 2022] POST /identity/v3/projects => generated 367 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:44:40.701211 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518cf157-d35d-42d1-86d6-e92e7bab2339 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:40.711956 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518cf157-d35d-42d1-86d6-e92e7bab2339 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:40.736086 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:40.777020 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19172d7-0250-429d-9152-0d0a1d69481d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:40.777724 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19172d7-0250-429d-9152-0d0a1d69481d admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:40.778012 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19172d7-0250-429d-9152-0d0a1d69481d admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:40.778294 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19172d7-0250-429d-9152-0d0a1d69481d admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:40.779052 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19172d7-0250-429d-9152-0d0a1d69481d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:40.781142 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19172d7-0250-429d-9152-0d0a1d69481d admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:40.804593 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1041/2088] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:44:40 2022] POST /identity/v3/users => generated 284 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:44:40.805784 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c9c99-e959-4ed8-9bd7-aaf53fb51d0c admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:40.816936 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c9c99-e959-4ed8-9bd7-aaf53fb51d0c admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:40.828123 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ab6eb-c65d-4b9d-92a9-5538bb4bb193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:40.828123 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ab6eb-c65d-4b9d-92a9-5538bb4bb193 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:40.828123 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ab6eb-c65d-4b9d-92a9-5538bb4bb193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:40.828123 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ab6eb-c65d-4b9d-92a9-5538bb4bb193 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:40.828757 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0ab6eb-c65d-4b9d-92a9-5538bb4bb193 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:40.828757 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0ab6eb-c65d-4b9d-92a9-5538bb4bb193 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:40.838133 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1048/2089] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:44:40.841565 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19172d7-0250-429d-9152-0d0a1d69481d admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:40.852386 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19172d7-0250-429d-9152-0d0a1d69481d admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:40.857307 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fb0439-73aa-485a-8158-0ab60ffd4e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:40.857859 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb0439-73aa-485a-8158-0ab60ffd4e2e admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:40.858148 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb0439-73aa-485a-8158-0ab60ffd4e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:40.858430 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb0439-73aa-485a-8158-0ab60ffd4e2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:40.859720 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fb0439-73aa-485a-8158-0ab60ffd4e2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:40.861342 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fb0439-73aa-485a-8158-0ab60ffd4e2e admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:40.871930 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1042/2090] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:44:40.875850 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ab6eb-c65d-4b9d-92a9-5538bb4bb193 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:40.886287 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ab6eb-c65d-4b9d-92a9-5538bb4bb193 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:40.891252 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91007768-20b7-4160-a9aa-d4f00651d58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:40.892149 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91007768-20b7-4160-a9aa-d4f00651d58c admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:40.892435 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91007768-20b7-4160-a9aa-d4f00651d58c admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:40.892722 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91007768-20b7-4160-a9aa-d4f00651d58c admin admin] PATH_INFO: `/v3/projects/c7fb1bb7e71e42b298b9852e2c1cf9fc/users/8c9d88a5863340b4ba44a01af0d00c32/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:40.910071 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91007768-20b7-4160-a9aa-d4f00651d58c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7fb1bb7e71e42b298b9852e2c1cf9fc, user_id=8c9d88a5863340b4ba44a01af0d00c32, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:40.912761 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91007768-20b7-4160-a9aa-d4f00651d58c admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:40.934213 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1049/2091] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:44:40 2022] PUT /identity/v3/projects/c7fb1bb7e71e42b298b9852e2c1cf9fc/users/8c9d88a5863340b4ba44a01af0d00c32/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:40.944919 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3728215-03db-4fe5-8770-a1a91c04c946 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:40.945337 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3728215-03db-4fe5-8770-a1a91c04c946 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:40.945721 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3728215-03db-4fe5-8770-a1a91c04c946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:40.980464 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-b3728215-03db-4fe5-8770-a1a91c04c946 None None] MFA Rules not processed for user `8c9d88a5863340b4ba44a01af0d00c32`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:40.981796 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-b3728215-03db-4fe5-8770-a1a91c04c946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:41.053025 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1043/2092] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:40 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:44:41.057602 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91007768-20b7-4160-a9aa-d4f00651d58c admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:41.125785 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91007768-20b7-4160-a9aa-d4f00651d58c admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:41.130914 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eef1cc7-5907-4e15-a097-66130107bf51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:41.131474 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eef1cc7-5907-4e15-a097-66130107bf51 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:41.131781 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eef1cc7-5907-4e15-a097-66130107bf51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:41.132040 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eef1cc7-5907-4e15-a097-66130107bf51 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:41.132740 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eef1cc7-5907-4e15-a097-66130107bf51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:41.134744 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eef1cc7-5907-4e15-a097-66130107bf51 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:41.160927 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1050/2093] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:44:41 2022] POST /identity/v3/projects => generated 367 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:44:41.165033 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3728215-03db-4fe5-8770-a1a91c04c946 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:41.176347 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3728215-03db-4fe5-8770-a1a91c04c946 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:41.232259 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d368ddc-9308-4a4a-a0c6-fb8d1f6ead2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:41.232927 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d368ddc-9308-4a4a-a0c6-fb8d1f6ead2d admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:41.233308 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d368ddc-9308-4a4a-a0c6-fb8d1f6ead2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:41.233532 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d368ddc-9308-4a4a-a0c6-fb8d1f6ead2d admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:41.234272 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d368ddc-9308-4a4a-a0c6-fb8d1f6ead2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:41.236202 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d368ddc-9308-4a4a-a0c6-fb8d1f6ead2d admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:41.255606 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1044/2094] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:44:41 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:44:41.259818 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eef1cc7-5907-4e15-a097-66130107bf51 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:41.270903 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eef1cc7-5907-4e15-a097-66130107bf51 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:41.275598 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c601f4-27d7-4298-81ac-ba6d099aea99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:41.276258 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c601f4-27d7-4298-81ac-ba6d099aea99 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:41.276545 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c601f4-27d7-4298-81ac-ba6d099aea99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:41.276825 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c601f4-27d7-4298-81ac-ba6d099aea99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:41.277603 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c601f4-27d7-4298-81ac-ba6d099aea99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:41.279213 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c601f4-27d7-4298-81ac-ba6d099aea99 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:41.285035 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:41.291389 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1051/2095] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:44:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:44:41.295388 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d368ddc-9308-4a4a-a0c6-fb8d1f6ead2d admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:41.305510 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d368ddc-9308-4a4a-a0c6-fb8d1f6ead2d admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:41.310121 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b4c9b-373a-4b39-897a-c77c15be144c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:41.310945 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b4c9b-373a-4b39-897a-c77c15be144c admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:41.311229 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b4c9b-373a-4b39-897a-c77c15be144c admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:41.311527 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b4c9b-373a-4b39-897a-c77c15be144c admin admin] PATH_INFO: `/v3/projects/1d78a0c6e73a4fc29819758099701984/users/b3556d9edce7446892fb10444b560975/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:41.325761 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825b4c9b-373a-4b39-897a-c77c15be144c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d78a0c6e73a4fc29819758099701984, user_id=b3556d9edce7446892fb10444b560975, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:41.328923 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825b4c9b-373a-4b39-897a-c77c15be144c admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:41.336381 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:41.342234 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1045/2096] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:44:41 2022] PUT /identity/v3/projects/1d78a0c6e73a4fc29819758099701984/users/b3556d9edce7446892fb10444b560975/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:41.346861 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c601f4-27d7-4298-81ac-ba6d099aea99 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:41.356750 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c601f4-27d7-4298-81ac-ba6d099aea99 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:41.412584 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38205907-f431-48c9-b486-b89f0b888505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:41.413454 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38205907-f431-48c9-b486-b89f0b888505 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:41.413747 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38205907-f431-48c9-b486-b89f0b888505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:41.414030 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38205907-f431-48c9-b486-b89f0b888505 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:41.414764 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38205907-f431-48c9-b486-b89f0b888505 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:41.416384 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38205907-f431-48c9-b486-b89f0b888505 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:41.426751 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1052/2097] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:44:41 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:44:41.430956 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b4c9b-373a-4b39-897a-c77c15be144c admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:41.441355 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b4c9b-373a-4b39-897a-c77c15be144c admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:41.446106 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c671098-5bbb-4ecb-927c-e5f38d109dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:41.447018 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c671098-5bbb-4ecb-927c-e5f38d109dcf admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:41.447307 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c671098-5bbb-4ecb-927c-e5f38d109dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:41.447595 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c671098-5bbb-4ecb-927c-e5f38d109dcf admin admin] PATH_INFO: `/v3/projects/1d78a0c6e73a4fc29819758099701984/users/b3556d9edce7446892fb10444b560975/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:41.449784 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c671098-5bbb-4ecb-927c-e5f38d109dcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d78a0c6e73a4fc29819758099701984, user_id=b3556d9edce7446892fb10444b560975, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:41.452281 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c671098-5bbb-4ecb-927c-e5f38d109dcf admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:41.463207 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1046/2098] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:44:41 2022] PUT /identity/v3/projects/1d78a0c6e73a4fc29819758099701984/users/b3556d9edce7446892fb10444b560975/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:41.474836 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f469e2c-e34d-4bbc-9da4-f9e41d0af927 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:41.475131 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f469e2c-e34d-4bbc-9da4-f9e41d0af927 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:41.475419 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f469e2c-e34d-4bbc-9da4-f9e41d0af927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:41.500981 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-9f469e2c-e34d-4bbc-9da4-f9e41d0af927 None None] MFA Rules not processed for user `b3556d9edce7446892fb10444b560975`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:41.502220 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-9f469e2c-e34d-4bbc-9da4-f9e41d0af927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:41.577006 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1053/2099] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:41 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:44:41.588460 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c671098-5bbb-4ecb-927c-e5f38d109dcf admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:41.590265 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-9c671098-5bbb-4ecb-927c-e5f38d109dcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:41.646745 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c671098-5bbb-4ecb-927c-e5f38d109dcf admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:41.650354 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a2ff40-408d-40cd-bc72-0b63c3111369 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:41.650999 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2ff40-408d-40cd-bc72-0b63c3111369 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:41.651285 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2ff40-408d-40cd-bc72-0b63c3111369 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:41.651566 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2ff40-408d-40cd-bc72-0b63c3111369 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:41.660629 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a2ff40-408d-40cd-bc72-0b63c3111369 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:41.662105 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a2ff40-408d-40cd-bc72-0b63c3111369 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:41.672453 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1047/2100] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:44:41 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:44:41.762688 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f469e2c-e34d-4bbc-9da4-f9e41d0af927 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:41.774442 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f469e2c-e34d-4bbc-9da4-f9e41d0af927 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:41.778697 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f226bc9-50cb-4fe7-9cfb-37742304eb75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:41.779433 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f226bc9-50cb-4fe7-9cfb-37742304eb75 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:41.779787 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f226bc9-50cb-4fe7-9cfb-37742304eb75 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:41.780197 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f226bc9-50cb-4fe7-9cfb-37742304eb75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:41.848511 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f226bc9-50cb-4fe7-9cfb-37742304eb75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:41.850141 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f226bc9-50cb-4fe7-9cfb-37742304eb75 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:41.864886 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1054/2101] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:44:41 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:44:42.851507 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a2ff40-408d-40cd-bc72-0b63c3111369 service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:42.853121 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-01a2ff40-408d-40cd-bc72-0b63c3111369 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:42.923852 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a2ff40-408d-40cd-bc72-0b63c3111369 service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:42.928615 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfc40c4-f05b-48ef-b5e4-e29cf5163395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:42.929549 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc40c4-f05b-48ef-b5e4-e29cf5163395 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:42.929833 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc40c4-f05b-48ef-b5e4-e29cf5163395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:42.930116 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc40c4-f05b-48ef-b5e4-e29cf5163395 admin admin] PATH_INFO: `/v3/users/26bcad09cb7749d780d151d3d2862795` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:42.931323 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfc40c4-f05b-48ef-b5e4-e29cf5163395 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26bcad09cb7749d780d151d3d2862795)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:42.933306 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfc40c4-f05b-48ef-b5e4-e29cf5163395 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:42.981556 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0bfc40c4-f05b-48ef-b5e4-e29cf5163395 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26bcad09cb7749d780d151d3d2862795'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:42.982218 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0bfc40c4-f05b-48ef-b5e4-e29cf5163395 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26bcad09cb7749d780d151d3d2862795'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:42.990491 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0bfc40c4-f05b-48ef-b5e4-e29cf5163395 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26bcad09cb7749d780d151d3d2862795'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:43.003607 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0bfc40c4-f05b-48ef-b5e4-e29cf5163395 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26bcad09cb7749d780d151d3d2862795'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:43.004304 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0bfc40c4-f05b-48ef-b5e4-e29cf5163395 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26bcad09cb7749d780d151d3d2862795'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:43.017694 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1048/2102] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:44:42 2022] DELETE /identity/v3/users/26bcad09cb7749d780d151d3d2862795 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:43.258401 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f226bc9-50cb-4fe7-9cfb-37742304eb75 service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:43.269900 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f226bc9-50cb-4fe7-9cfb-37742304eb75 service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:43.297616 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:43.339345 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb67e1-3e37-448a-8ce4-71e99e8faf4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:43.340203 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb67e1-3e37-448a-8ce4-71e99e8faf4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:43.340493 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb67e1-3e37-448a-8ce4-71e99e8faf4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:43.340803 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb67e1-3e37-448a-8ce4-71e99e8faf4f admin admin] PATH_INFO: `/v3/projects/16022d299eda48e6bf4675055870a140` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:43.342037 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eb67e1-3e37-448a-8ce4-71e99e8faf4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=16022d299eda48e6bf4675055870a140)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:43.344468 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eb67e1-3e37-448a-8ce4-71e99e8faf4f admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:43.390618 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-80eb67e1-3e37-448a-8ce4-71e99e8faf4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16022d299eda48e6bf4675055870a140', 'request_id': 'req-80eb67e1-3e37-448a-8ce4-71e99e8faf4f'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:43.396262 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-80eb67e1-3e37-448a-8ce4-71e99e8faf4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16022d299eda48e6bf4675055870a140', 'request_id': 'req-80eb67e1-3e37-448a-8ce4-71e99e8faf4f'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:43.402729 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-80eb67e1-3e37-448a-8ce4-71e99e8faf4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16022d299eda48e6bf4675055870a140', 'request_id': 'req-80eb67e1-3e37-448a-8ce4-71e99e8faf4f'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:43.415239 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-80eb67e1-3e37-448a-8ce4-71e99e8faf4f admin admin] The token cache is being invalidate because project 16022d299eda48e6bf4675055870a140 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:44:43.415530 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-80eb67e1-3e37-448a-8ce4-71e99e8faf4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:43.416180 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-80eb67e1-3e37-448a-8ce4-71e99e8faf4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:43.417879 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1055/2103] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:44:43 2022] DELETE /identity/v3/projects/16022d299eda48e6bf4675055870a140 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:43.423515 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45ca63-9801-49a5-8c58-6161dea92079 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:43.423813 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45ca63-9801-49a5-8c58-6161dea92079 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:43.424100 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45ca63-9801-49a5-8c58-6161dea92079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:43.436535 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:43.442949 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-9d45ca63-9801-49a5-8c58-6161dea92079 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:43.444305 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-9d45ca63-9801-49a5-8c58-6161dea92079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:43.504616 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1049/2104] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:44:43.512628 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf67cc4-02a6-4f47-87e3-019ad50fa602 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:43.512924 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf67cc4-02a6-4f47-87e3-019ad50fa602 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:43.513240 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf67cc4-02a6-4f47-87e3-019ad50fa602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:43.532228 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-dbf67cc4-02a6-4f47-87e3-019ad50fa602 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:43.533473 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-dbf67cc4-02a6-4f47-87e3-019ad50fa602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:43.541212 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1056/2105] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:44:43.545471 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d45ca63-9801-49a5-8c58-6161dea92079 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:43.555123 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d45ca63-9801-49a5-8c58-6161dea92079 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:43.559956 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955bb1b-4691-4a73-8f30-f6de9185df23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:43.560615 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955bb1b-4691-4a73-8f30-f6de9185df23 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:43.560901 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955bb1b-4691-4a73-8f30-f6de9185df23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:43.561210 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955bb1b-4691-4a73-8f30-f6de9185df23 admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:43.561982 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4955bb1b-4691-4a73-8f30-f6de9185df23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:44:43.562288 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4955bb1b-4691-4a73-8f30-f6de9185df23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:43.563890 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4955bb1b-4691-4a73-8f30-f6de9185df23 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:43.575558 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1050/2106] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:44:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:44:43.580383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf67cc4-02a6-4f47-87e3-019ad50fa602 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:43.589621 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf67cc4-02a6-4f47-87e3-019ad50fa602 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:43.595019 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64404af9-9052-449f-8fd5-2a23155f532a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:43.595724 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64404af9-9052-449f-8fd5-2a23155f532a admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:43.596013 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64404af9-9052-449f-8fd5-2a23155f532a admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:43.596296 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64404af9-9052-449f-8fd5-2a23155f532a admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:43.597039 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64404af9-9052-449f-8fd5-2a23155f532a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:43.599025 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64404af9-9052-449f-8fd5-2a23155f532a admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:43.624344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1057/2107] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:44:43 2022] POST /identity/v3/projects => generated 375 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:44:43.628507 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955bb1b-4691-4a73-8f30-f6de9185df23 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:43.638869 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955bb1b-4691-4a73-8f30-f6de9185df23 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:43.695410 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ed7cc-2f10-4de2-82df-fdf54209ae7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:43.696089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ed7cc-2f10-4de2-82df-fdf54209ae7b admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:43.696381 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ed7cc-2f10-4de2-82df-fdf54209ae7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:43.696691 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ed7cc-2f10-4de2-82df-fdf54209ae7b admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:43.697553 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ed7cc-2f10-4de2-82df-fdf54209ae7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:43.699488 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ed7cc-2f10-4de2-82df-fdf54209ae7b admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:43.720767 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1051/2108] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:44:43 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:44:43.726030 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64404af9-9052-449f-8fd5-2a23155f532a admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:43.736479 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64404af9-9052-449f-8fd5-2a23155f532a admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:43.741058 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649ae83d-f8a4-4a66-8575-9d566524992d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:43.741749 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ae83d-f8a4-4a66-8575-9d566524992d admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:43.742040 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ae83d-f8a4-4a66-8575-9d566524992d admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:43.742324 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ae83d-f8a4-4a66-8575-9d566524992d admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:43.743060 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649ae83d-f8a4-4a66-8575-9d566524992d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:43.744685 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649ae83d-f8a4-4a66-8575-9d566524992d admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:43.755560 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1058/2109] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:44:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:44:43.760710 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ed7cc-2f10-4de2-82df-fdf54209ae7b admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:43.771586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ed7cc-2f10-4de2-82df-fdf54209ae7b admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:43.776716 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a0db11-e5a2-41f4-8647-4420022070d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:43.777116 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0db11-e5a2-41f4-8647-4420022070d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:43.777443 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0db11-e5a2-41f4-8647-4420022070d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:43.777783 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0db11-e5a2-41f4-8647-4420022070d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:43.778523 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a0db11-e5a2-41f4-8647-4420022070d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:43.780130 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a0db11-e5a2-41f4-8647-4420022070d0 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:43.792187 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1052/2110] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:44:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:44:43.795304 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649ae83d-f8a4-4a66-8575-9d566524992d admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:43.805714 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649ae83d-f8a4-4a66-8575-9d566524992d admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:43.810776 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791913d-a50f-4252-b013-afd827afb495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:43.811705 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791913d-a50f-4252-b013-afd827afb495 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:43.811942 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791913d-a50f-4252-b013-afd827afb495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:43.812478 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791913d-a50f-4252-b013-afd827afb495 admin admin] PATH_INFO: `/v3/projects/fd5d1150a9ed483bbd298b896bcf7f46/users/4077d35bf1044be79bfaa013e13c530e/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:43.827617 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f791913d-a50f-4252-b013-afd827afb495 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd5d1150a9ed483bbd298b896bcf7f46, user_id=4077d35bf1044be79bfaa013e13c530e, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:43.830912 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f791913d-a50f-4252-b013-afd827afb495 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:43.843512 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1059/2111] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:44:43 2022] PUT /identity/v3/projects/fd5d1150a9ed483bbd298b896bcf7f46/users/4077d35bf1044be79bfaa013e13c530e/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:43.850390 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65075688-86df-4c2d-9baf-4b4a30a9a175 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:43.850683 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65075688-86df-4c2d-9baf-4b4a30a9a175 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:43.850970 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65075688-86df-4c2d-9baf-4b4a30a9a175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:43.879052 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-65075688-86df-4c2d-9baf-4b4a30a9a175 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:43.879052 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-65075688-86df-4c2d-9baf-4b4a30a9a175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:43.945949 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1053/2112] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:44:43.955966 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791913d-a50f-4252-b013-afd827afb495 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:43.957769 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-f791913d-a50f-4252-b013-afd827afb495 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:44.010742 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:44.017706 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791913d-a50f-4252-b013-afd827afb495 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:44.022467 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cdf459-21c0-48c0-a774-8786b3d2b4fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:44.023268 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cdf459-21c0-48c0-a774-8786b3d2b4fc service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:44.023554 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cdf459-21c0-48c0-a774-8786b3d2b4fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:44.023832 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cdf459-21c0-48c0-a774-8786b3d2b4fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:44.032921 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cdf459-21c0-48c0-a774-8786b3d2b4fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:44.034402 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cdf459-21c0-48c0-a774-8786b3d2b4fc service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:44.043731 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1060/2113] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:44:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:44:48.215120 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65075688-86df-4c2d-9baf-4b4a30a9a175 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:48.217110 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-65075688-86df-4c2d-9baf-4b4a30a9a175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:48.237498 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65075688-86df-4c2d-9baf-4b4a30a9a175 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:48.246916 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba919238-72b8-4d73-abba-ca56e6f5ac0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:48.250673 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba919238-72b8-4d73-abba-ca56e6f5ac0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:48.251728 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba919238-72b8-4d73-abba-ca56e6f5ac0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:48.252547 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba919238-72b8-4d73-abba-ca56e6f5ac0c admin admin] PATH_INFO: `/v3/users/8c9d88a5863340b4ba44a01af0d00c32` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:48.254436 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba919238-72b8-4d73-abba-ca56e6f5ac0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c9d88a5863340b4ba44a01af0d00c32)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:48.257556 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba919238-72b8-4d73-abba-ca56e6f5ac0c admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:48.265368 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:48.347218 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-ba919238-72b8-4d73-abba-ca56e6f5ac0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c9d88a5863340b4ba44a01af0d00c32'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:48.348283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-ba919238-72b8-4d73-abba-ca56e6f5ac0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c9d88a5863340b4ba44a01af0d00c32'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:48.362135 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-ba919238-72b8-4d73-abba-ca56e6f5ac0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c9d88a5863340b4ba44a01af0d00c32'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:48.381259 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-ba919238-72b8-4d73-abba-ca56e6f5ac0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c9d88a5863340b4ba44a01af0d00c32'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:48.382797 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-ba919238-72b8-4d73-abba-ca56e6f5ac0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c9d88a5863340b4ba44a01af0d00c32'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:48.403267 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a15cc77-7934-459c-826c-839998fb18ac None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:48.403819 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a15cc77-7934-459c-826c-839998fb18ac None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:48.403819 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a15cc77-7934-459c-826c-839998fb18ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:48.407406 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1054/2114] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:44:48 2022] DELETE /identity/v3/users/8c9d88a5863340b4ba44a01af0d00c32 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:48.429001 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-1a15cc77-7934-459c-826c-839998fb18ac None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:48.430287 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-1a15cc77-7934-459c-826c-839998fb18ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:48.628606 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1061/2115] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:44:48.638907 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba919238-72b8-4d73-abba-ca56e6f5ac0c admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:48.640537 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ba919238-72b8-4d73-abba-ca56e6f5ac0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:48.758856 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba919238-72b8-4d73-abba-ca56e6f5ac0c admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:48.771903 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae7402-edb3-48bf-bd34-e91e72e569a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:48.772770 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae7402-edb3-48bf-bd34-e91e72e569a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:48.773066 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae7402-edb3-48bf-bd34-e91e72e569a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:48.773459 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae7402-edb3-48bf-bd34-e91e72e569a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:48.815648 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ae7402-edb3-48bf-bd34-e91e72e569a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:48.817342 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ae7402-edb3-48bf-bd34-e91e72e569a4 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:48.835589 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1055/2116] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:44:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:44:49.258605 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e142a7-a2f1-4559-add3-cfe89db60306 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:49.259347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e142a7-a2f1-4559-add3-cfe89db60306 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:49.259836 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e142a7-a2f1-4559-add3-cfe89db60306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:49.310234 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-b6e142a7-a2f1-4559-add3-cfe89db60306 None None] MFA Rules not processed for user `4077d35bf1044be79bfaa013e13c530e`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:49.312065 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b6e142a7-a2f1-4559-add3-cfe89db60306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:49.399845 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae7402-edb3-48bf-bd34-e91e72e569a4 service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:49.401448 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-b0ae7402-edb3-48bf-bd34-e91e72e569a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:49.416951 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae7402-edb3-48bf-bd34-e91e72e569a4 service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:49.423457 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1062/2117] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:44:49.441581 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e142a7-a2f1-4559-add3-cfe89db60306 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:49.443549 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b6e142a7-a2f1-4559-add3-cfe89db60306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:49.455446 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541bf669-f742-4e6b-b5bd-0d4e23b095bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:49.456416 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541bf669-f742-4e6b-b5bd-0d4e23b095bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:49.456821 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541bf669-f742-4e6b-b5bd-0d4e23b095bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:49.457190 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541bf669-f742-4e6b-b5bd-0d4e23b095bd admin admin] PATH_INFO: `/v3/projects/c7fb1bb7e71e42b298b9852e2c1cf9fc` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:49.459935 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541bf669-f742-4e6b-b5bd-0d4e23b095bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7fb1bb7e71e42b298b9852e2c1cf9fc)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:49.462164 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541bf669-f742-4e6b-b5bd-0d4e23b095bd admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:49.530874 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-541bf669-f742-4e6b-b5bd-0d4e23b095bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7fb1bb7e71e42b298b9852e2c1cf9fc', 'request_id': 'req-541bf669-f742-4e6b-b5bd-0d4e23b095bd'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:49.541919 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-541bf669-f742-4e6b-b5bd-0d4e23b095bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7fb1bb7e71e42b298b9852e2c1cf9fc', 'request_id': 'req-541bf669-f742-4e6b-b5bd-0d4e23b095bd'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:49.544082 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e142a7-a2f1-4559-add3-cfe89db60306 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:49.549544 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-541bf669-f742-4e6b-b5bd-0d4e23b095bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7fb1bb7e71e42b298b9852e2c1cf9fc', 'request_id': 'req-541bf669-f742-4e6b-b5bd-0d4e23b095bd'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:49.564951 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:49.566524 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-541bf669-f742-4e6b-b5bd-0d4e23b095bd admin admin] The token cache is being invalidate because project c7fb1bb7e71e42b298b9852e2c1cf9fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:44:49.566855 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-541bf669-f742-4e6b-b5bd-0d4e23b095bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:49.568248 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-541bf669-f742-4e6b-b5bd-0d4e23b095bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:49.574710 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1056/2118] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:44:49 2022] DELETE /identity/v3/projects/c7fb1bb7e71e42b298b9852e2c1cf9fc => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:49.575656 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541bf669-f742-4e6b-b5bd-0d4e23b095bd admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:49.577584 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-541bf669-f742-4e6b-b5bd-0d4e23b095bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:49.598543 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:49.650207 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13c5195-cf4e-4612-93d2-b24874620100 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:49.650775 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13c5195-cf4e-4612-93d2-b24874620100 service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:49.651064 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13c5195-cf4e-4612-93d2-b24874620100 service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:49.651695 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13c5195-cf4e-4612-93d2-b24874620100 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:49.655733 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b13c5195-cf4e-4612-93d2-b24874620100 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:49.683422 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541bf669-f742-4e6b-b5bd-0d4e23b095bd admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:49.689042 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e5f9fb-e58d-4cc0-9325-1d48f5d0b8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:49.690176 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5f9fb-e58d-4cc0-9325-1d48f5d0b8c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:49.690505 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5f9fb-e58d-4cc0-9325-1d48f5d0b8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:49.690795 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5f9fb-e58d-4cc0-9325-1d48f5d0b8c3 admin admin] PATH_INFO: `/v3/users/b3556d9edce7446892fb10444b560975` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:49.692109 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e5f9fb-e58d-4cc0-9325-1d48f5d0b8c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3556d9edce7446892fb10444b560975)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:49.694220 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e5f9fb-e58d-4cc0-9325-1d48f5d0b8c3 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:49.733366 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13c5195-cf4e-4612-93d2-b24874620100 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:49.735377 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13c5195-cf4e-4612-93d2-b24874620100 service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:49.745097 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1063/2119] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:44:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:44:49.750351 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-e6e5f9fb-e58d-4cc0-9325-1d48f5d0b8c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3556d9edce7446892fb10444b560975'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:49.751061 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-e6e5f9fb-e58d-4cc0-9325-1d48f5d0b8c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3556d9edce7446892fb10444b560975'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:49.759378 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-e6e5f9fb-e58d-4cc0-9325-1d48f5d0b8c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3556d9edce7446892fb10444b560975'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:49.773503 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-e6e5f9fb-e58d-4cc0-9325-1d48f5d0b8c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3556d9edce7446892fb10444b560975'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:49.774286 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-e6e5f9fb-e58d-4cc0-9325-1d48f5d0b8c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3556d9edce7446892fb10444b560975'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:49.789653 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1057/2120] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:44:49 2022] DELETE /identity/v3/users/b3556d9edce7446892fb10444b560975 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:50.105982 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13c5195-cf4e-4612-93d2-b24874620100 service nova] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:50.119143 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13c5195-cf4e-4612-93d2-b24874620100 service nova] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:50.190761 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600d791f-5194-41a4-ac7c-5f23155a3561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:50.195360 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600d791f-5194-41a4-ac7c-5f23155a3561 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:50.195701 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600d791f-5194-41a4-ac7c-5f23155a3561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:50.195984 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600d791f-5194-41a4-ac7c-5f23155a3561 admin admin] PATH_INFO: `/v3/projects/1d78a0c6e73a4fc29819758099701984` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:50.199114 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600d791f-5194-41a4-ac7c-5f23155a3561 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d78a0c6e73a4fc29819758099701984)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:50.201579 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600d791f-5194-41a4-ac7c-5f23155a3561 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:50.258338 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-600d791f-5194-41a4-ac7c-5f23155a3561 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d78a0c6e73a4fc29819758099701984', 'request_id': 'req-600d791f-5194-41a4-ac7c-5f23155a3561'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:50.263516 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:50.265538 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-600d791f-5194-41a4-ac7c-5f23155a3561 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d78a0c6e73a4fc29819758099701984', 'request_id': 'req-600d791f-5194-41a4-ac7c-5f23155a3561'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:50.272609 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-600d791f-5194-41a4-ac7c-5f23155a3561 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d78a0c6e73a4fc29819758099701984', 'request_id': 'req-600d791f-5194-41a4-ac7c-5f23155a3561'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:50.287006 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-600d791f-5194-41a4-ac7c-5f23155a3561 admin admin] The token cache is being invalidate because project 1d78a0c6e73a4fc29819758099701984 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:44:50.287540 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-600d791f-5194-41a4-ac7c-5f23155a3561 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:50.288311 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-600d791f-5194-41a4-ac7c-5f23155a3561 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:50.290144 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1064/2121] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:44:50 2022] DELETE /identity/v3/projects/1d78a0c6e73a4fc29819758099701984 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:50.295019 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540fd95-32a5-4090-ac0a-00468d10e220 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:50.295317 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540fd95-32a5-4090-ac0a-00468d10e220 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:50.295651 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540fd95-32a5-4090-ac0a-00468d10e220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:50.319281 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-4540fd95-32a5-4090-ac0a-00468d10e220 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:50.320522 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-4540fd95-32a5-4090-ac0a-00468d10e220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:50.389602 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1058/2122] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:44:50.397547 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba492a-4f50-42d5-820c-a6b180ba3e2c None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:50.397700 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba492a-4f50-42d5-820c-a6b180ba3e2c None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:50.397985 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba492a-4f50-42d5-820c-a6b180ba3e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:50.418145 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-19ba492a-4f50-42d5-820c-a6b180ba3e2c None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:50.419448 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-19ba492a-4f50-42d5-820c-a6b180ba3e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:50.432933 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4540fd95-32a5-4090-ac0a-00468d10e220 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:50.433464 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1065/2123] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:44:50.445015 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4540fd95-32a5-4090-ac0a-00468d10e220 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:50.453913 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb0b95-fea0-4139-820a-f7a1b544b752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:50.454603 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb0b95-fea0-4139-820a-f7a1b544b752 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:50.454890 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb0b95-fea0-4139-820a-f7a1b544b752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:50.455169 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb0b95-fea0-4139-820a-f7a1b544b752 admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:50.456595 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfb0b95-fea0-4139-820a-f7a1b544b752 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:44:50.456958 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfb0b95-fea0-4139-820a-f7a1b544b752 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:50.458521 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfb0b95-fea0-4139-820a-f7a1b544b752 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:50.476131 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ba492a-4f50-42d5-820c-a6b180ba3e2c None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:50.479178 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1059/2124] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:44:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:44:50.488602 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ba492a-4f50-42d5-820c-a6b180ba3e2c None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:50.493742 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdedcc8-f49b-4432-8b99-52a54ca811c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:50.494557 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdedcc8-f49b-4432-8b99-52a54ca811c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:50.494853 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdedcc8-f49b-4432-8b99-52a54ca811c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:50.495133 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdedcc8-f49b-4432-8b99-52a54ca811c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:50.495968 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdedcc8-f49b-4432-8b99-52a54ca811c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:50.498023 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdedcc8-f49b-4432-8b99-52a54ca811c0 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:50.545218 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1066/2125] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:44:50 2022] POST /identity/v3/projects => generated 381 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:44:50.550020 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb0b95-fea0-4139-820a-f7a1b544b752 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:50.566276 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb0b95-fea0-4139-820a-f7a1b544b752 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:50.607072 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:50.684035 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140db122-0a2d-4e87-88e9-91f349587d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:50.684726 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140db122-0a2d-4e87-88e9-91f349587d2c admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:50.685017 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140db122-0a2d-4e87-88e9-91f349587d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:50.685363 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140db122-0a2d-4e87-88e9-91f349587d2c admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:50.686298 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140db122-0a2d-4e87-88e9-91f349587d2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:50.688342 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140db122-0a2d-4e87-88e9-91f349587d2c admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:50.712692 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1060/2126] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:44:50 2022] POST /identity/v3/users => generated 291 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:44:50.716620 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdedcc8-f49b-4432-8b99-52a54ca811c0 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:50.731952 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdedcc8-f49b-4432-8b99-52a54ca811c0 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:50.743798 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140db122-0a2d-4e87-88e9-91f349587d2c admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:50.745335 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-140db122-0a2d-4e87-88e9-91f349587d2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:50.747103 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde6273-fa3c-4683-ae87-0b882e3b349a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:50.748031 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde6273-fa3c-4683-ae87-0b882e3b349a admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:50.748388 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde6273-fa3c-4683-ae87-0b882e3b349a admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:50.748732 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde6273-fa3c-4683-ae87-0b882e3b349a admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:50.750313 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cde6273-fa3c-4683-ae87-0b882e3b349a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:50.753216 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cde6273-fa3c-4683-ae87-0b882e3b349a admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:50.759929 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140db122-0a2d-4e87-88e9-91f349587d2c admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:50.765726 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44f641f-67c6-436b-8277-e9dd84308524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:50.766746 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f641f-67c6-436b-8277-e9dd84308524 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:50.767103 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f641f-67c6-436b-8277-e9dd84308524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:50.767468 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f641f-67c6-436b-8277-e9dd84308524 admin admin] PATH_INFO: `/v3/users/17c5cbd044fb4fd6a3592a5b75cfc6cc` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:50.768764 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44f641f-67c6-436b-8277-e9dd84308524 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17c5cbd044fb4fd6a3592a5b75cfc6cc)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:50.769715 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1067/2127] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:44:50 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:44:50.770906 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44f641f-67c6-436b-8277-e9dd84308524 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:50.774429 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde6273-fa3c-4683-ae87-0b882e3b349a admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:50.788946 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde6273-fa3c-4683-ae87-0b882e3b349a admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:50.796246 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b873bd-581d-4016-b449-8e3050341a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:50.797825 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b873bd-581d-4016-b449-8e3050341a8d admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:50.798557 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b873bd-581d-4016-b449-8e3050341a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:50.799018 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b873bd-581d-4016-b449-8e3050341a8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:50.800557 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b873bd-581d-4016-b449-8e3050341a8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:50.804379 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b873bd-581d-4016-b449-8e3050341a8d admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:50.823152 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1068/2128] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:44:50 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:44:50.825214 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b873bd-581d-4016-b449-8e3050341a8d admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:50.836021 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-c44f641f-67c6-436b-8277-e9dd84308524 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17c5cbd044fb4fd6a3592a5b75cfc6cc'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:50.836825 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-c44f641f-67c6-436b-8277-e9dd84308524 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17c5cbd044fb4fd6a3592a5b75cfc6cc'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:50.845487 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b873bd-581d-4016-b449-8e3050341a8d admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:50.847321 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-c44f641f-67c6-436b-8277-e9dd84308524 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17c5cbd044fb4fd6a3592a5b75cfc6cc'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:50.851624 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2970e7-e1f4-40da-98fc-e5839574b049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:50.852292 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2970e7-e1f4-40da-98fc-e5839574b049 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:50.852665 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2970e7-e1f4-40da-98fc-e5839574b049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:50.853041 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2970e7-e1f4-40da-98fc-e5839574b049 admin admin] PATH_INFO: `/v3/projects/c5ed921bc8a84afbaccd928efb76fd67/users/2a29d8426f4a43529e43dee17c240d38/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:50.866647 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-c44f641f-67c6-436b-8277-e9dd84308524 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17c5cbd044fb4fd6a3592a5b75cfc6cc'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:50.867525 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-c44f641f-67c6-436b-8277-e9dd84308524 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17c5cbd044fb4fd6a3592a5b75cfc6cc'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:50.875221 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2970e7-e1f4-40da-98fc-e5839574b049 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5ed921bc8a84afbaccd928efb76fd67, user_id=2a29d8426f4a43529e43dee17c240d38, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:50.879342 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2970e7-e1f4-40da-98fc-e5839574b049 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:50.892478 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1061/2129] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:44:50 2022] DELETE /identity/v3/users/17c5cbd044fb4fd6a3592a5b75cfc6cc => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:50.901543 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48e53d-76e5-494e-bb9b-e38a02286c9e None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:50.903127 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48e53d-76e5-494e-bb9b-e38a02286c9e None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:50.904195 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48e53d-76e5-494e-bb9b-e38a02286c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:50.905514 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1069/2130] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:44:50 2022] PUT /identity/v3/projects/c5ed921bc8a84afbaccd928efb76fd67/users/2a29d8426f4a43529e43dee17c240d38/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:44:50.919593 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327a067-4e7f-49cc-b0de-3234808347c0 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:50.919881 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327a067-4e7f-49cc-b0de-3234808347c0 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:50.920165 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327a067-4e7f-49cc-b0de-3234808347c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:50.937851 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-ea48e53d-76e5-494e-bb9b-e38a02286c9e None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:50.939159 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ea48e53d-76e5-494e-bb9b-e38a02286c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:50.978562 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-1327a067-4e7f-49cc-b0de-3234808347c0 None None] MFA Rules not processed for user `2a29d8426f4a43529e43dee17c240d38`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:50.980806 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-1327a067-4e7f-49cc-b0de-3234808347c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:51.081731 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1062/2131] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:44:51.094851 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea48e53d-76e5-494e-bb9b-e38a02286c9e None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:51.096330 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ea48e53d-76e5-494e-bb9b-e38a02286c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:51.100339 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1070/2132] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:44:51.101465 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1327a067-4e7f-49cc-b0de-3234808347c0 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:51.103053 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-1327a067-4e7f-49cc-b0de-3234808347c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:51.124327 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1327a067-4e7f-49cc-b0de-3234808347c0 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:51.131631 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb8bc1-9bc9-4d76-b28a-92d7780ae2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:51.132537 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb8bc1-9bc9-4d76-b28a-92d7780ae2a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:51.133270 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb8bc1-9bc9-4d76-b28a-92d7780ae2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:51.133497 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:51.133630 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb8bc1-9bc9-4d76-b28a-92d7780ae2a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:51.134402 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feb8bc1-9bc9-4d76-b28a-92d7780ae2a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:51.136374 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feb8bc1-9bc9-4d76-b28a-92d7780ae2a4 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:51.190010 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea48e53d-76e5-494e-bb9b-e38a02286c9e None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:51.191961 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1071/2133] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:44:51 2022] POST /identity/v3/projects => generated 383 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:44:51.196213 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb8bc1-9bc9-4d76-b28a-92d7780ae2a4 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:51.258384 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:51.304815 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb8bc1-9bc9-4d76-b28a-92d7780ae2a4 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:51.306969 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176be638-206b-4372-bf34-d39f3e8f4117 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:51.307637 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176be638-206b-4372-bf34-d39f3e8f4117 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:51.307917 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176be638-206b-4372-bf34-d39f3e8f4117 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:51.308190 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176be638-206b-4372-bf34-d39f3e8f4117 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:51.319604 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeffaec-35de-4339-94c3-5b5c2a876ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:51.320358 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeffaec-35de-4339-94c3-5b5c2a876ae2 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:51.320654 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeffaec-35de-4339-94c3-5b5c2a876ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:51.320918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeffaec-35de-4339-94c3-5b5c2a876ae2 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:51.321666 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeffaec-35de-4339-94c3-5b5c2a876ae2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:51.323862 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeffaec-35de-4339-94c3-5b5c2a876ae2 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:51.326967 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176be638-206b-4372-bf34-d39f3e8f4117 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:51.329568 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176be638-206b-4372-bf34-d39f3e8f4117 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:51.346208 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1063/2134] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:44:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:44:51.354046 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1072/2135] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:44:51 2022] POST /identity/v3/users => generated 298 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:44:51.354955 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176be638-206b-4372-bf34-d39f3e8f4117 service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:51.368074 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176be638-206b-4372-bf34-d39f3e8f4117 service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:51.373884 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14c39fb-1041-41e6-9e0f-4baed60e5a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:51.374584 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c39fb-1041-41e6-9e0f-4baed60e5a4b admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:51.375385 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c39fb-1041-41e6-9e0f-4baed60e5a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:51.375678 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c39fb-1041-41e6-9e0f-4baed60e5a4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:51.376427 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14c39fb-1041-41e6-9e0f-4baed60e5a4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:51.378528 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14c39fb-1041-41e6-9e0f-4baed60e5a4b admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:51.394855 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1064/2136] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:44:51 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:44:51.399750 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeffaec-35de-4339-94c3-5b5c2a876ae2 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:51.412586 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeffaec-35de-4339-94c3-5b5c2a876ae2 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:51.430489 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0056ea6a-431f-48ba-946a-08becbf52b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:51.433336 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056ea6a-431f-48ba-946a-08becbf52b2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:51.433336 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056ea6a-431f-48ba-946a-08becbf52b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:51.433336 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056ea6a-431f-48ba-946a-08becbf52b2e admin admin] PATH_INFO: `/v3/projects/b18180d0a9e846aea48103992ec83e0d/users/432ea80018d74dcebb5511792da5cb13/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:51.458198 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0056ea6a-431f-48ba-946a-08becbf52b2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b18180d0a9e846aea48103992ec83e0d, user_id=432ea80018d74dcebb5511792da5cb13, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:51.462201 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0056ea6a-431f-48ba-946a-08becbf52b2e admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:51.481329 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1073/2137] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:44:51 2022] PUT /identity/v3/projects/b18180d0a9e846aea48103992ec83e0d/users/432ea80018d74dcebb5511792da5cb13/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:51.485474 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14c39fb-1041-41e6-9e0f-4baed60e5a4b admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:51.501621 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14c39fb-1041-41e6-9e0f-4baed60e5a4b admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:51.596827 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e47da2-39f4-4f9a-80a0-fd159f0f217d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:51.598700 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e47da2-39f4-4f9a-80a0-fd159f0f217d admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:51.598994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e47da2-39f4-4f9a-80a0-fd159f0f217d admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:51.599281 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e47da2-39f4-4f9a-80a0-fd159f0f217d admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:51.600039 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e47da2-39f4-4f9a-80a0-fd159f0f217d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:51.601690 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e47da2-39f4-4f9a-80a0-fd159f0f217d admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:51.616095 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1065/2138] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:44:51 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:44:51.617473 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0056ea6a-431f-48ba-946a-08becbf52b2e admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:51.628744 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0056ea6a-431f-48ba-946a-08becbf52b2e admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:51.637859 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e37ae5-a8e5-4d92-adc8-46262761897f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:51.639547 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e37ae5-a8e5-4d92-adc8-46262761897f admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:51.640854 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e37ae5-a8e5-4d92-adc8-46262761897f admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:51.641528 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e37ae5-a8e5-4d92-adc8-46262761897f admin admin] PATH_INFO: `/v3/projects/b18180d0a9e846aea48103992ec83e0d/users/432ea80018d74dcebb5511792da5cb13/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:51.644884 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e37ae5-a8e5-4d92-adc8-46262761897f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b18180d0a9e846aea48103992ec83e0d, user_id=432ea80018d74dcebb5511792da5cb13, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:51.648169 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e37ae5-a8e5-4d92-adc8-46262761897f admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:51.665563 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1074/2139] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:44:51 2022] PUT /identity/v3/projects/b18180d0a9e846aea48103992ec83e0d/users/432ea80018d74dcebb5511792da5cb13/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:51.677664 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bef471-c0da-4884-844b-cdc956535761 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:51.677967 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bef471-c0da-4884-844b-cdc956535761 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:51.678263 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bef471-c0da-4884-844b-cdc956535761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:51.712713 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-72bef471-c0da-4884-844b-cdc956535761 None None] MFA Rules not processed for user `432ea80018d74dcebb5511792da5cb13`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:51.715223 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-72bef471-c0da-4884-844b-cdc956535761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:51.760758 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914f0d6-13a9-43bc-8dfc-95a1445a1c23 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:51.761059 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914f0d6-13a9-43bc-8dfc-95a1445a1c23 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:51.761422 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914f0d6-13a9-43bc-8dfc-95a1445a1c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:51.797586 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-e914f0d6-13a9-43bc-8dfc-95a1445a1c23 None None] MFA Rules not processed for user `4077d35bf1044be79bfaa013e13c530e`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:51.799163 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e914f0d6-13a9-43bc-8dfc-95a1445a1c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:51.830185 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:51.921453 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1066/2140] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:51 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:44:51.929603 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bef471-c0da-4884-844b-cdc956535761 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:51.933668 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-72bef471-c0da-4884-844b-cdc956535761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:51.944432 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1075/2141] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:44:51.945137 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e914f0d6-13a9-43bc-8dfc-95a1445a1c23 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:51.947594 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e914f0d6-13a9-43bc-8dfc-95a1445a1c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:52.072919 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e914f0d6-13a9-43bc-8dfc-95a1445a1c23 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:52.078875 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d0a89c-bc2c-4db1-9f33-a395ee07f556 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:52.079552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d0a89c-bc2c-4db1-9f33-a395ee07f556 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:52.079845 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d0a89c-bc2c-4db1-9f33-a395ee07f556 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:52.080125 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d0a89c-bc2c-4db1-9f33-a395ee07f556 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:52.111814 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d0a89c-bc2c-4db1-9f33-a395ee07f556 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:52.113695 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d0a89c-bc2c-4db1-9f33-a395ee07f556 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:52.136038 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bef471-c0da-4884-844b-cdc956535761 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:52.142812 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00e172-858b-4acb-ae13-e7d36109a4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:52.143924 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00e172-858b-4acb-ae13-e7d36109a4ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:52.144706 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00e172-858b-4acb-ae13-e7d36109a4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:52.145255 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00e172-858b-4acb-ae13-e7d36109a4ed admin admin] PATH_INFO: `/v3/projects/3bd0d1dadadb480ab97b93fb7a5b2562` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:52.147818 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb00e172-858b-4acb-ae13-e7d36109a4ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bd0d1dadadb480ab97b93fb7a5b2562)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:52.154369 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1076/2142] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:44:51 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:44:52.156749 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb00e172-858b-4acb-ae13-e7d36109a4ed admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:52.248031 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-cb00e172-858b-4acb-ae13-e7d36109a4ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bd0d1dadadb480ab97b93fb7a5b2562', 'request_id': 'req-cb00e172-858b-4acb-ae13-e7d36109a4ed'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:52.255674 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-cb00e172-858b-4acb-ae13-e7d36109a4ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bd0d1dadadb480ab97b93fb7a5b2562', 'request_id': 'req-cb00e172-858b-4acb-ae13-e7d36109a4ed'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:52.264145 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-cb00e172-858b-4acb-ae13-e7d36109a4ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bd0d1dadadb480ab97b93fb7a5b2562', 'request_id': 'req-cb00e172-858b-4acb-ae13-e7d36109a4ed'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:52.305815 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-cb00e172-858b-4acb-ae13-e7d36109a4ed admin admin] The token cache is being invalidate because project 3bd0d1dadadb480ab97b93fb7a5b2562 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:44:52.306253 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-cb00e172-858b-4acb-ae13-e7d36109a4ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:52.307618 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-cb00e172-858b-4acb-ae13-e7d36109a4ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:52.311662 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1067/2143] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:44:51 2022] DELETE /identity/v3/projects/3bd0d1dadadb480ab97b93fb7a5b2562 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:52.316824 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7e141-fa96-4a2f-b791-c07bd7a2a21a None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:52.317127 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7e141-fa96-4a2f-b791-c07bd7a2a21a None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:52.317478 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7e141-fa96-4a2f-b791-c07bd7a2a21a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:52.352088 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00e172-858b-4acb-ae13-e7d36109a4ed admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:52.354432 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-cb00e172-858b-4acb-ae13-e7d36109a4ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:52.359851 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-0cc7e141-fa96-4a2f-b791-c07bd7a2a21a None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:52.362511 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-0cc7e141-fa96-4a2f-b791-c07bd7a2a21a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:52.452033 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:52.498974 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1077/2144] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:44:52.501561 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcf00b-4f6e-4e4c-b0e0-6ac99f0fbc4c None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:52.501820 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcf00b-4f6e-4e4c-b0e0-6ac99f0fbc4c None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:52.502104 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcf00b-4f6e-4e4c-b0e0-6ac99f0fbc4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:52.521398 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00e172-858b-4acb-ae13-e7d36109a4ed admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:52.529387 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3873cb-67cc-4148-980c-d7641af5363f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:52.530002 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3873cb-67cc-4148-980c-d7641af5363f service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:52.530929 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3873cb-67cc-4148-980c-d7641af5363f service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:52.531276 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3873cb-67cc-4148-980c-d7641af5363f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:52.535857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ff3873cb-67cc-4148-980c-d7641af5363f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:52.544209 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-c0dcf00b-4f6e-4e4c-b0e0-6ac99f0fbc4c None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:52.548134 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-c0dcf00b-4f6e-4e4c-b0e0-6ac99f0fbc4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:52.566107 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1078/2145] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:44:52.567142 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dcf00b-4f6e-4e4c-b0e0-6ac99f0fbc4c None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:52.582975 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dcf00b-4f6e-4e4c-b0e0-6ac99f0fbc4c None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:52.595806 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f781e81-94ec-4abc-8467-e6df0467f603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:52.598327 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f781e81-94ec-4abc-8467-e6df0467f603 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:52.598629 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f781e81-94ec-4abc-8467-e6df0467f603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:52.600303 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f781e81-94ec-4abc-8467-e6df0467f603 admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:52.601694 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f781e81-94ec-4abc-8467-e6df0467f603 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:44:52.602805 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f781e81-94ec-4abc-8467-e6df0467f603 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:52.604726 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f781e81-94ec-4abc-8467-e6df0467f603 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:52.628517 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1079/2146] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:44:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:44:52.633296 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f781e81-94ec-4abc-8467-e6df0467f603 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:52.648997 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f781e81-94ec-4abc-8467-e6df0467f603 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:52.654043 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:52.658267 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dde26f-114f-4a89-9e4d-9ded5496502e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:52.663403 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dde26f-114f-4a89-9e4d-9ded5496502e admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:52.665196 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dde26f-114f-4a89-9e4d-9ded5496502e admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:52.665910 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dde26f-114f-4a89-9e4d-9ded5496502e admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:52.666943 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dde26f-114f-4a89-9e4d-9ded5496502e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:52.669308 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dde26f-114f-4a89-9e4d-9ded5496502e admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:52.720490 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1080/2147] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:44:52 2022] POST /identity/v3/projects => generated 383 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:44:52.732438 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dde26f-114f-4a89-9e4d-9ded5496502e admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:52.751816 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3873cb-67cc-4148-980c-d7641af5363f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:52.755113 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3873cb-67cc-4148-980c-d7641af5363f service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:52.776398 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1068/2148] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:44:52 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:44:52.919316 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dde26f-114f-4a89-9e4d-9ded5496502e admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:52.941503 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f5b76-c69a-4fa6-af8f-26a16cae4d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:52.942304 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f5b76-c69a-4fa6-af8f-26a16cae4d1a admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:52.942475 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f5b76-c69a-4fa6-af8f-26a16cae4d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:52.942755 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f5b76-c69a-4fa6-af8f-26a16cae4d1a admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:52.944252 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809f5b76-c69a-4fa6-af8f-26a16cae4d1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:52.947059 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809f5b76-c69a-4fa6-af8f-26a16cae4d1a admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:52.979193 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1081/2149] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:44:52 2022] POST /identity/v3/users => generated 292 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:44:52.983722 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3873cb-67cc-4148-980c-d7641af5363f service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:53.004553 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3873cb-67cc-4148-980c-d7641af5363f service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:53.011666 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a5e21-56b1-42d2-9a79-a3d4c40cc1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:53.012606 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a5e21-56b1-42d2-9a79-a3d4c40cc1ce admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:53.013147 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a5e21-56b1-42d2-9a79-a3d4c40cc1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:53.014200 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a5e21-56b1-42d2-9a79-a3d4c40cc1ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:53.014432 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9a5e21-56b1-42d2-9a79-a3d4c40cc1ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:53.017012 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9a5e21-56b1-42d2-9a79-a3d4c40cc1ce admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:53.039413 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1069/2150] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:44:52 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:44:53.043906 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f5b76-c69a-4fa6-af8f-26a16cae4d1a admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:53.056282 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f5b76-c69a-4fa6-af8f-26a16cae4d1a admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:53.068791 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fc9979-507b-4dc6-8edf-80c36a368292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:53.069525 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc9979-507b-4dc6-8edf-80c36a368292 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:53.069831 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc9979-507b-4dc6-8edf-80c36a368292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:53.070172 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc9979-507b-4dc6-8edf-80c36a368292 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:53.071039 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fc9979-507b-4dc6-8edf-80c36a368292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:53.081201 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fc9979-507b-4dc6-8edf-80c36a368292 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:53.125807 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1082/2151] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:44:53 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:44:53.130758 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a5e21-56b1-42d2-9a79-a3d4c40cc1ce admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:53.149057 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a5e21-56b1-42d2-9a79-a3d4c40cc1ce admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:53.183914 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d8bb63-b481-4785-87a8-0050c998c541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:53.185117 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d8bb63-b481-4785-87a8-0050c998c541 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:53.186120 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d8bb63-b481-4785-87a8-0050c998c541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:53.186468 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d8bb63-b481-4785-87a8-0050c998c541 admin admin] PATH_INFO: `/v3/projects/d4e27cee4a8745bbad913909b6f7dd4a/users/fc94434a2665492c80ce156f599b64cc/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:53.216195 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d8bb63-b481-4785-87a8-0050c998c541 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4e27cee4a8745bbad913909b6f7dd4a, user_id=fc94434a2665492c80ce156f599b64cc, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:53.222320 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d8bb63-b481-4785-87a8-0050c998c541 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:53.255303 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1070/2152] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:44:53 2022] PUT /identity/v3/projects/d4e27cee4a8745bbad913909b6f7dd4a/users/fc94434a2665492c80ce156f599b64cc/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:53.270336 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97dc8e-e584-4a05-967d-c0efa4912a66 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:53.271429 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97dc8e-e584-4a05-967d-c0efa4912a66 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:53.272316 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97dc8e-e584-4a05-967d-c0efa4912a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:53.325805 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-be97dc8e-e584-4a05-967d-c0efa4912a66 None None] MFA Rules not processed for user `fc94434a2665492c80ce156f599b64cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:53.327139 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-be97dc8e-e584-4a05-967d-c0efa4912a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:53.327336 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d8bb63-b481-4785-87a8-0050c998c541 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:53.330285 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-61d8bb63-b481-4785-87a8-0050c998c541 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:53.437312 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d8bb63-b481-4785-87a8-0050c998c541 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:53.449684 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b159f5cf-ffcb-48a3-8fa2-79da11a9f6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:53.450503 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159f5cf-ffcb-48a3-8fa2-79da11a9f6e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:53.451019 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159f5cf-ffcb-48a3-8fa2-79da11a9f6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:53.451349 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159f5cf-ffcb-48a3-8fa2-79da11a9f6e3 admin admin] PATH_INFO: `/v3/users/2a29d8426f4a43529e43dee17c240d38` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:53.453650 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b159f5cf-ffcb-48a3-8fa2-79da11a9f6e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a29d8426f4a43529e43dee17c240d38)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:53.456561 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b159f5cf-ffcb-48a3-8fa2-79da11a9f6e3 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:53.473332 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1083/2153] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:44:53.474633 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be97dc8e-e584-4a05-967d-c0efa4912a66 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:53.486405 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:53.501475 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be97dc8e-e584-4a05-967d-c0efa4912a66 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:53.513981 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5551642-4369-43eb-a8ec-882a5ae1046b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:53.517504 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5551642-4369-43eb-a8ec-882a5ae1046b admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:53.518323 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5551642-4369-43eb-a8ec-882a5ae1046b admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:53.518833 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5551642-4369-43eb-a8ec-882a5ae1046b admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:53.519791 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5551642-4369-43eb-a8ec-882a5ae1046b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:53.522473 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5551642-4369-43eb-a8ec-882a5ae1046b admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:53.544100 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b159f5cf-ffcb-48a3-8fa2-79da11a9f6e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a29d8426f4a43529e43dee17c240d38'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:53.545362 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b159f5cf-ffcb-48a3-8fa2-79da11a9f6e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a29d8426f4a43529e43dee17c240d38'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:53.559042 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b159f5cf-ffcb-48a3-8fa2-79da11a9f6e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a29d8426f4a43529e43dee17c240d38'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:53.575827 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1084/2154] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:44:53 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:44:53.575827 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5551642-4369-43eb-a8ec-882a5ae1046b admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:53.587133 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b159f5cf-ffcb-48a3-8fa2-79da11a9f6e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a29d8426f4a43529e43dee17c240d38'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:53.588527 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b159f5cf-ffcb-48a3-8fa2-79da11a9f6e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a29d8426f4a43529e43dee17c240d38'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:53.614461 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1071/2155] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:44:53 2022] DELETE /identity/v3/users/2a29d8426f4a43529e43dee17c240d38 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:53.624343 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89682deb-6ab8-4521-821b-ae3cf9fbf951 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:53.625371 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89682deb-6ab8-4521-821b-ae3cf9fbf951 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:53.625727 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89682deb-6ab8-4521-821b-ae3cf9fbf951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:53.649350 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-89682deb-6ab8-4521-821b-ae3cf9fbf951 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:53.650826 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-89682deb-6ab8-4521-821b-ae3cf9fbf951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:53.678258 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:53.697253 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5551642-4369-43eb-a8ec-882a5ae1046b admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:53.767906 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1072/2156] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:44:53.783717 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89682deb-6ab8-4521-821b-ae3cf9fbf951 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:53.785836 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-89682deb-6ab8-4521-821b-ae3cf9fbf951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:53.793102 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79bb3a5-fe1a-4926-95e8-19b0a4806d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:53.795264 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79bb3a5-fe1a-4926-95e8-19b0a4806d6b admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:53.795991 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79bb3a5-fe1a-4926-95e8-19b0a4806d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:53.796479 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79bb3a5-fe1a-4926-95e8-19b0a4806d6b admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:53.797847 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79bb3a5-fe1a-4926-95e8-19b0a4806d6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:53.802061 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79bb3a5-fe1a-4926-95e8-19b0a4806d6b admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:53.833838 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1085/2157] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:44:53 2022] POST /identity/v3/users => generated 293 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:44:53.837761 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79bb3a5-fe1a-4926-95e8-19b0a4806d6b admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:53.839574 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-c79bb3a5-fe1a-4926-95e8-19b0a4806d6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:53.856362 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79bb3a5-fe1a-4926-95e8-19b0a4806d6b admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:53.868278 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22ae0d3-80f2-4538-8e83-df7a3c912d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:53.869034 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89682deb-6ab8-4521-821b-ae3cf9fbf951 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:53.869284 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ae0d3-80f2-4538-8e83-df7a3c912d86 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:53.869652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ae0d3-80f2-4538-8e83-df7a3c912d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:53.870009 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ae0d3-80f2-4538-8e83-df7a3c912d86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:53.870866 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22ae0d3-80f2-4538-8e83-df7a3c912d86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:53.872654 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22ae0d3-80f2-4538-8e83-df7a3c912d86 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:53.881941 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bd7bae-53da-4312-a8c4-ac5241b5ec04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:53.882634 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd7bae-53da-4312-a8c4-ac5241b5ec04 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:53.885511 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd7bae-53da-4312-a8c4-ac5241b5ec04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:53.887355 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd7bae-53da-4312-a8c4-ac5241b5ec04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:53.896727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1086/2158] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:44:53 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:44:53.902130 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22ae0d3-80f2-4538-8e83-df7a3c912d86 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:53.910443 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bd7bae-53da-4312-a8c4-ac5241b5ec04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:53.912963 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bd7bae-53da-4312-a8c4-ac5241b5ec04 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:53.932143 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22ae0d3-80f2-4538-8e83-df7a3c912d86 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:53.938682 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e53ac4c-f49f-498f-9f4a-ad5eb656116f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:53.940049 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e53ac4c-f49f-498f-9f4a-ad5eb656116f admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:53.940464 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e53ac4c-f49f-498f-9f4a-ad5eb656116f admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:53.940894 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e53ac4c-f49f-498f-9f4a-ad5eb656116f admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:53.941939 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e53ac4c-f49f-498f-9f4a-ad5eb656116f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:53.944195 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e53ac4c-f49f-498f-9f4a-ad5eb656116f admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:53.953577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1073/2159] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:44:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 16:44:53.960628 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1087/2160] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:44:53 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:44:53.964291 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bd7bae-53da-4312-a8c4-ac5241b5ec04 service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:53.993488 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bd7bae-53da-4312-a8c4-ac5241b5ec04 service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:54.013195 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5982db5b-0d3b-43d2-837b-d0f2de9b79ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:54.015639 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5982db5b-0d3b-43d2-837b-d0f2de9b79ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:54.016256 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5982db5b-0d3b-43d2-837b-d0f2de9b79ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:54.016704 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5982db5b-0d3b-43d2-837b-d0f2de9b79ad admin admin] PATH_INFO: `/v3/projects/9e4278f8faf948068f633e00c8826a52/users/818f0dfcd232420d921088e932b7d3d7/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:54.037905 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5982db5b-0d3b-43d2-837b-d0f2de9b79ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e4278f8faf948068f633e00c8826a52, user_id=818f0dfcd232420d921088e932b7d3d7, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:54.040754 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5982db5b-0d3b-43d2-837b-d0f2de9b79ad admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:54.079559 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1074/2161] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:44:53 2022] PUT /identity/v3/projects/9e4278f8faf948068f633e00c8826a52/users/818f0dfcd232420d921088e932b7d3d7/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:54.091098 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cbbdb8-f8e9-47db-93b5-62f92ad4843e None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:54.093226 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cbbdb8-f8e9-47db-93b5-62f92ad4843e None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:54.093764 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cbbdb8-f8e9-47db-93b5-62f92ad4843e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:54.153281 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-e3cbbdb8-f8e9-47db-93b5-62f92ad4843e None None] MFA Rules not processed for user `818f0dfcd232420d921088e932b7d3d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:54.154818 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e3cbbdb8-f8e9-47db-93b5-62f92ad4843e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:54.224802 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:54.251522 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1088/2162] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:44:54.260714 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5982db5b-0d3b-43d2-837b-d0f2de9b79ad admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:54.338544 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5982db5b-0d3b-43d2-837b-d0f2de9b79ad admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:54.345048 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d0cff-251d-4131-9786-7ad498a63b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:54.345824 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d0cff-251d-4131-9786-7ad498a63b6f admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:54.346114 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d0cff-251d-4131-9786-7ad498a63b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:54.346392 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d0cff-251d-4131-9786-7ad498a63b6f admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:54.347130 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97d0cff-251d-4131-9786-7ad498a63b6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:54.349091 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97d0cff-251d-4131-9786-7ad498a63b6f admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:54.386779 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1075/2163] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:44:54 2022] POST /identity/v3/projects => generated 381 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:44:54.396011 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cbbdb8-f8e9-47db-93b5-62f92ad4843e None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:54.419905 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d0cff-251d-4131-9786-7ad498a63b6f admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:54.422346 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-c97d0cff-251d-4131-9786-7ad498a63b6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:54.427244 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cbbdb8-f8e9-47db-93b5-62f92ad4843e None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:54.466865 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:54.540688 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d0cff-251d-4131-9786-7ad498a63b6f admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:54.547533 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d39e33-94c2-4ffd-8874-bfbb0a6f1cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:54.548486 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d39e33-94c2-4ffd-8874-bfbb0a6f1cef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:54.548792 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d39e33-94c2-4ffd-8874-bfbb0a6f1cef admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:54.549082 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d39e33-94c2-4ffd-8874-bfbb0a6f1cef admin admin] PATH_INFO: `/v3/projects/c5ed921bc8a84afbaccd928efb76fd67` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:54.550478 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d39e33-94c2-4ffd-8874-bfbb0a6f1cef admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5ed921bc8a84afbaccd928efb76fd67)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:54.552431 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d39e33-94c2-4ffd-8874-bfbb0a6f1cef admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:54.577379 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6657cc-2044-4bbf-9a43-aae4589557a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:54.579268 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6657cc-2044-4bbf-9a43-aae4589557a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:54.579758 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6657cc-2044-4bbf-9a43-aae4589557a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:54.580023 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6657cc-2044-4bbf-9a43-aae4589557a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:54.580916 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6657cc-2044-4bbf-9a43-aae4589557a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:54.583462 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6657cc-2044-4bbf-9a43-aae4589557a1 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:54.651149 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1089/2164] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:44:54 2022] POST /identity/v3/users => generated 297 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:44:54.654775 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6657cc-2044-4bbf-9a43-aae4589557a1 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:54.656590 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-51d39e33-94c2-4ffd-8874-bfbb0a6f1cef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5ed921bc8a84afbaccd928efb76fd67', 'request_id': 'req-51d39e33-94c2-4ffd-8874-bfbb0a6f1cef'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:54.673481 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-51d39e33-94c2-4ffd-8874-bfbb0a6f1cef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5ed921bc8a84afbaccd928efb76fd67', 'request_id': 'req-51d39e33-94c2-4ffd-8874-bfbb0a6f1cef'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:54.673896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6657cc-2044-4bbf-9a43-aae4589557a1 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:54.682117 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-51d39e33-94c2-4ffd-8874-bfbb0a6f1cef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5ed921bc8a84afbaccd928efb76fd67', 'request_id': 'req-51d39e33-94c2-4ffd-8874-bfbb0a6f1cef'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:54.711953 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-51d39e33-94c2-4ffd-8874-bfbb0a6f1cef admin admin] The token cache is being invalidate because project c5ed921bc8a84afbaccd928efb76fd67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:44:54.712481 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-51d39e33-94c2-4ffd-8874-bfbb0a6f1cef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:54.713231 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-51d39e33-94c2-4ffd-8874-bfbb0a6f1cef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:54.715079 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1076/2165] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:44:54 2022] DELETE /identity/v3/projects/c5ed921bc8a84afbaccd928efb76fd67 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:54.719730 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d39e33-94c2-4ffd-8874-bfbb0a6f1cef admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:54.732970 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-51d39e33-94c2-4ffd-8874-bfbb0a6f1cef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:54.804811 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e66299-bcb7-489a-a491-2b5f9e6bfe7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:54.805665 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e66299-bcb7-489a-a491-2b5f9e6bfe7d admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:54.806571 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e66299-bcb7-489a-a491-2b5f9e6bfe7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:54.806860 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e66299-bcb7-489a-a491-2b5f9e6bfe7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:54.807642 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e66299-bcb7-489a-a491-2b5f9e6bfe7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:54.810822 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e66299-bcb7-489a-a491-2b5f9e6bfe7d admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:54.830332 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1090/2166] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:44:54 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:44:54.831374 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e66299-bcb7-489a-a491-2b5f9e6bfe7d admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:54.834410 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-c1e66299-bcb7-489a-a491-2b5f9e6bfe7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:54.857452 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e66299-bcb7-489a-a491-2b5f9e6bfe7d admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:54.864311 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f8d934-1ac7-4984-a3d4-bcb36ae12b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:54.865810 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f8d934-1ac7-4984-a3d4-bcb36ae12b1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:54.866113 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f8d934-1ac7-4984-a3d4-bcb36ae12b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:54.866552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f8d934-1ac7-4984-a3d4-bcb36ae12b1a admin admin] PATH_INFO: `/v3/projects/061467f24c364080a76433e2f48e6077/users/6c655f01b28a46d692b1ff9fa79b4e07/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:54.897961 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f8d934-1ac7-4984-a3d4-bcb36ae12b1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=061467f24c364080a76433e2f48e6077, user_id=6c655f01b28a46d692b1ff9fa79b4e07, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:54.898592 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d39e33-94c2-4ffd-8874-bfbb0a6f1cef admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:54.901029 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f8d934-1ac7-4984-a3d4-bcb36ae12b1a admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:54.905053 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c0744f-a1ba-4dae-82b0-7aaa9f541946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:54.906021 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0744f-a1ba-4dae-82b0-7aaa9f541946 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:54.906476 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0744f-a1ba-4dae-82b0-7aaa9f541946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:54.906890 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0744f-a1ba-4dae-82b0-7aaa9f541946 admin admin] PATH_INFO: `/v3/users/432ea80018d74dcebb5511792da5cb13` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:54.914232 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c0744f-a1ba-4dae-82b0-7aaa9f541946 admin admin] RBAC: Authorizing `identity:delete_user(user_id=432ea80018d74dcebb5511792da5cb13)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:54.918732 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c0744f-a1ba-4dae-82b0-7aaa9f541946 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:54.924667 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1091/2167] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:44:54 2022] PUT /identity/v3/projects/061467f24c364080a76433e2f48e6077/users/6c655f01b28a46d692b1ff9fa79b4e07/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:54.931157 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f8d934-1ac7-4984-a3d4-bcb36ae12b1a admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:54.994741 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:54.994923 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-e5c0744f-a1ba-4dae-82b0-7aaa9f541946 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '432ea80018d74dcebb5511792da5cb13'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:54.996164 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-e5c0744f-a1ba-4dae-82b0-7aaa9f541946 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '432ea80018d74dcebb5511792da5cb13'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:55.015163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-e5c0744f-a1ba-4dae-82b0-7aaa9f541946 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '432ea80018d74dcebb5511792da5cb13'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:55.017881 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:55.020656 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f8d934-1ac7-4984-a3d4-bcb36ae12b1a admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:55.035879 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-e5c0744f-a1ba-4dae-82b0-7aaa9f541946 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '432ea80018d74dcebb5511792da5cb13'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:55.036104 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806d5a97-15ed-42b1-9567-85b8bca60b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:55.036609 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-e5c0744f-a1ba-4dae-82b0-7aaa9f541946 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '432ea80018d74dcebb5511792da5cb13'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:55.037350 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d5a97-15ed-42b1-9567-85b8bca60b78 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:55.037735 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d5a97-15ed-42b1-9567-85b8bca60b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:55.037920 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d5a97-15ed-42b1-9567-85b8bca60b78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:55.038951 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806d5a97-15ed-42b1-9567-85b8bca60b78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:55.041372 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806d5a97-15ed-42b1-9567-85b8bca60b78 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:55.057258 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1092/2168] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:44:54 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:44:55.057453 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1077/2169] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:44:54 2022] DELETE /identity/v3/users/432ea80018d74dcebb5511792da5cb13 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:55.059699 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806d5a97-15ed-42b1-9567-85b8bca60b78 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:55.061486 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-806d5a97-15ed-42b1-9567-85b8bca60b78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:55.075255 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806d5a97-15ed-42b1-9567-85b8bca60b78 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:55.136409 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d601d-1a4d-435f-83ec-b9eb92dfd696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:55.137289 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d601d-1a4d-435f-83ec-b9eb92dfd696 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:55.137581 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d601d-1a4d-435f-83ec-b9eb92dfd696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:55.137887 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d601d-1a4d-435f-83ec-b9eb92dfd696 admin admin] PATH_INFO: `/v3/projects/061467f24c364080a76433e2f48e6077/users/6c655f01b28a46d692b1ff9fa79b4e07/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:55.139973 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338d601d-1a4d-435f-83ec-b9eb92dfd696 admin admin] RBAC: Authorizing `identity:create_grant(project_id=061467f24c364080a76433e2f48e6077, user_id=6c655f01b28a46d692b1ff9fa79b4e07, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:55.146764 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338d601d-1a4d-435f-83ec-b9eb92dfd696 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:55.158926 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1093/2170] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:44:55 2022] PUT /identity/v3/projects/061467f24c364080a76433e2f48e6077/users/6c655f01b28a46d692b1ff9fa79b4e07/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:44:55.169220 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f9cbf-5a07-4b55-9484-fe85eae5289d None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:55.169633 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f9cbf-5a07-4b55-9484-fe85eae5289d None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:55.169836 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f9cbf-5a07-4b55-9484-fe85eae5289d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:55.204812 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-310f9cbf-5a07-4b55-9484-fe85eae5289d None None] MFA Rules not processed for user `6c655f01b28a46d692b1ff9fa79b4e07`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:55.206260 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-310f9cbf-5a07-4b55-9484-fe85eae5289d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:55.291344 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1078/2171] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:55 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:44:55.306059 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d601d-1a4d-435f-83ec-b9eb92dfd696 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:55.307805 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-338d601d-1a4d-435f-83ec-b9eb92dfd696 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:55.387093 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d601d-1a4d-435f-83ec-b9eb92dfd696 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:55.391325 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde30ce-655b-4323-8068-1a244c63bd78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:55.394655 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde30ce-655b-4323-8068-1a244c63bd78 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:55.394655 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde30ce-655b-4323-8068-1a244c63bd78 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:55.394655 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde30ce-655b-4323-8068-1a244c63bd78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:55.399994 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-cbde30ce-655b-4323-8068-1a244c63bd78 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:55.486928 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbde30ce-655b-4323-8068-1a244c63bd78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:55.488688 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbde30ce-655b-4323-8068-1a244c63bd78 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:55.500498 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1094/2172] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:44:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:44:55.570284 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f9cbf-5a07-4b55-9484-fe85eae5289d None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:55.571821 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-310f9cbf-5a07-4b55-9484-fe85eae5289d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:55.666472 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f9cbf-5a07-4b55-9484-fe85eae5289d None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:55.672810 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beea9f9-d0b6-46db-a6dc-1eda4af5ed6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:55.673824 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beea9f9-d0b6-46db-a6dc-1eda4af5ed6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:55.674039 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beea9f9-d0b6-46db-a6dc-1eda4af5ed6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:55.674322 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beea9f9-d0b6-46db-a6dc-1eda4af5ed6f admin admin] PATH_INFO: `/v3/projects/b18180d0a9e846aea48103992ec83e0d` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:55.676264 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beea9f9-d0b6-46db-a6dc-1eda4af5ed6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b18180d0a9e846aea48103992ec83e0d)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:55.678295 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beea9f9-d0b6-46db-a6dc-1eda4af5ed6f admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:55.749559 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8beea9f9-d0b6-46db-a6dc-1eda4af5ed6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b18180d0a9e846aea48103992ec83e0d', 'request_id': 'req-8beea9f9-d0b6-46db-a6dc-1eda4af5ed6f'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:55.756997 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8beea9f9-d0b6-46db-a6dc-1eda4af5ed6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b18180d0a9e846aea48103992ec83e0d', 'request_id': 'req-8beea9f9-d0b6-46db-a6dc-1eda4af5ed6f'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:55.758444 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:55.769303 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8beea9f9-d0b6-46db-a6dc-1eda4af5ed6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b18180d0a9e846aea48103992ec83e0d', 'request_id': 'req-8beea9f9-d0b6-46db-a6dc-1eda4af5ed6f'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:55.793568 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8beea9f9-d0b6-46db-a6dc-1eda4af5ed6f admin admin] The token cache is being invalidate because project b18180d0a9e846aea48103992ec83e0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:44:55.794231 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8beea9f9-d0b6-46db-a6dc-1eda4af5ed6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:55.794929 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8beea9f9-d0b6-46db-a6dc-1eda4af5ed6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:44:55.797842 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1079/2173] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:44:55 2022] DELETE /identity/v3/projects/b18180d0a9e846aea48103992ec83e0d => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:55.802796 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1daeaa-9a9c-4c06-b145-7b93e7d7e16d None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:55.803316 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1daeaa-9a9c-4c06-b145-7b93e7d7e16d None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:55.803519 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1daeaa-9a9c-4c06-b145-7b93e7d7e16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:55.844432 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-3f1daeaa-9a9c-4c06-b145-7b93e7d7e16d None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:55.844432 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-3f1daeaa-9a9c-4c06-b145-7b93e7d7e16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:55.887728 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:55.943771 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1095/2174] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:44:55.953107 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68a519-3168-468b-98d9-a2c30f9a7153 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:55.953551 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68a519-3168-468b-98d9-a2c30f9a7153 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:55.953801 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68a519-3168-468b-98d9-a2c30f9a7153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:55.982066 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-0b68a519-3168-468b-98d9-a2c30f9a7153 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:55.983474 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-0b68a519-3168-468b-98d9-a2c30f9a7153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:55.997962 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1080/2175] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:44:56.002488 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1daeaa-9a9c-4c06-b145-7b93e7d7e16d None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:56.015344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1daeaa-9a9c-4c06-b145-7b93e7d7e16d None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:56.026790 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7582fce2-3f95-4723-8b6e-4c941a4eecad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:56.027509 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582fce2-3f95-4723-8b6e-4c941a4eecad admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:56.027991 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582fce2-3f95-4723-8b6e-4c941a4eecad admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:56.028286 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582fce2-3f95-4723-8b6e-4c941a4eecad admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:56.029068 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7582fce2-3f95-4723-8b6e-4c941a4eecad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:44:56.029621 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7582fce2-3f95-4723-8b6e-4c941a4eecad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:56.032302 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7582fce2-3f95-4723-8b6e-4c941a4eecad admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:56.047201 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1096/2176] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:44:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:44:56.052310 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68a519-3168-468b-98d9-a2c30f9a7153 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:56.064471 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68a519-3168-468b-98d9-a2c30f9a7153 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:56.070785 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d0ef18-6f67-4c9a-947c-6773e1fdaa0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:56.071950 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0ef18-6f67-4c9a-947c-6773e1fdaa0c admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:56.072460 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0ef18-6f67-4c9a-947c-6773e1fdaa0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:56.074292 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0ef18-6f67-4c9a-947c-6773e1fdaa0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:56.075358 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d0ef18-6f67-4c9a-947c-6773e1fdaa0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:56.077443 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d0ef18-6f67-4c9a-947c-6773e1fdaa0c admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:56.126227 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1081/2177] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:44:56 2022] POST /identity/v3/projects => generated 399 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:44:56.130987 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7582fce2-3f95-4723-8b6e-4c941a4eecad admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:56.142381 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7582fce2-3f95-4723-8b6e-4c941a4eecad admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:56.219417 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a8c1a-407d-4c34-8971-6cd0b7a5163e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:56.220105 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a8c1a-407d-4c34-8971-6cd0b7a5163e admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:56.220507 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a8c1a-407d-4c34-8971-6cd0b7a5163e admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:56.220795 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a8c1a-407d-4c34-8971-6cd0b7a5163e admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:56.221538 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a8c1a-407d-4c34-8971-6cd0b7a5163e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:56.223503 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a8c1a-407d-4c34-8971-6cd0b7a5163e admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:56.247344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1097/2178] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:44:56 2022] POST /identity/v3/users => generated 300 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:44:56.248508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d0ef18-6f67-4c9a-947c-6773e1fdaa0c admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:56.264357 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d0ef18-6f67-4c9a-947c-6773e1fdaa0c admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:56.272304 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca44ebd-235a-4932-92b4-7c2106c4c52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:56.273224 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca44ebd-235a-4932-92b4-7c2106c4c52d admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:56.273695 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca44ebd-235a-4932-92b4-7c2106c4c52d admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:56.274185 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca44ebd-235a-4932-92b4-7c2106c4c52d admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:56.275024 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca44ebd-235a-4932-92b4-7c2106c4c52d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:56.276764 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca44ebd-235a-4932-92b4-7c2106c4c52d admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:56.292983 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1082/2179] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:44:56 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:44:56.294272 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a8c1a-407d-4c34-8971-6cd0b7a5163e admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:56.306436 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a8c1a-407d-4c34-8971-6cd0b7a5163e admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:56.311696 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a3c1c2-2b2b-4e37-b636-b2016bc48222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:56.312648 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3c1c2-2b2b-4e37-b636-b2016bc48222 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:56.313209 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3c1c2-2b2b-4e37-b636-b2016bc48222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:56.313569 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3c1c2-2b2b-4e37-b636-b2016bc48222 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:56.314555 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a3c1c2-2b2b-4e37-b636-b2016bc48222 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:56.316911 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a3c1c2-2b2b-4e37-b636-b2016bc48222 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:56.327576 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1098/2180] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:44:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:44:56.332328 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca44ebd-235a-4932-92b4-7c2106c4c52d admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:56.344450 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca44ebd-235a-4932-92b4-7c2106c4c52d admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:56.349976 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf9168d-cf44-4d7f-b63c-b799f5671ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:56.351504 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9168d-cf44-4d7f-b63c-b799f5671ed2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:56.351880 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9168d-cf44-4d7f-b63c-b799f5671ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:56.352263 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9168d-cf44-4d7f-b63c-b799f5671ed2 admin admin] PATH_INFO: `/v3/projects/47516a554956442a9cf0330aa3f5bb14/users/be97e1eab35e4245958d03a9a4f3d429/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:56.378902 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf9168d-cf44-4d7f-b63c-b799f5671ed2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47516a554956442a9cf0330aa3f5bb14, user_id=be97e1eab35e4245958d03a9a4f3d429, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:56.381600 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf9168d-cf44-4d7f-b63c-b799f5671ed2 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:56.401229 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1083/2181] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:44:56 2022] PUT /identity/v3/projects/47516a554956442a9cf0330aa3f5bb14/users/be97e1eab35e4245958d03a9a4f3d429/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:44:56.411148 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf73ea-f3ce-4992-8174-e49c9d32ba97 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:56.411787 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf73ea-f3ce-4992-8174-e49c9d32ba97 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:56.412305 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf73ea-f3ce-4992-8174-e49c9d32ba97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:56.455362 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-7ddf73ea-f3ce-4992-8174-e49c9d32ba97 None None] MFA Rules not processed for user `be97e1eab35e4245958d03a9a4f3d429`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:44:56.456619 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-7ddf73ea-f3ce-4992-8174-e49c9d32ba97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:56.542957 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1099/2182] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:44:56 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:44:56.551771 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf9168d-cf44-4d7f-b63c-b799f5671ed2 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:56.555067 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-8cf9168d-cf44-4d7f-b63c-b799f5671ed2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:56.655845 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf9168d-cf44-4d7f-b63c-b799f5671ed2 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:56.660431 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dbda2a-6f10-4fcc-8e6f-4bb8f0d95ff8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:56.661185 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dbda2a-6f10-4fcc-8e6f-4bb8f0d95ff8 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:56.661463 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dbda2a-6f10-4fcc-8e6f-4bb8f0d95ff8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:56.661765 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dbda2a-6f10-4fcc-8e6f-4bb8f0d95ff8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:56.671410 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dbda2a-6f10-4fcc-8e6f-4bb8f0d95ff8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:56.673254 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dbda2a-6f10-4fcc-8e6f-4bb8f0d95ff8 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:56.695659 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1084/2183] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:44:56 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:44:57.185067 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddf73ea-f3ce-4992-8174-e49c9d32ba97 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:57.187916 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-7ddf73ea-f3ce-4992-8174-e49c9d32ba97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:57.210963 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddf73ea-f3ce-4992-8174-e49c9d32ba97 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:57.214678 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6663da43-5f26-4b32-b71d-5164569c4acb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:57.215375 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663da43-5f26-4b32-b71d-5164569c4acb service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:57.215629 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663da43-5f26-4b32-b71d-5164569c4acb service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:57.215923 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663da43-5f26-4b32-b71d-5164569c4acb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:57.217900 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-6663da43-5f26-4b32-b71d-5164569c4acb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:57.259828 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:44:57.301519 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6663da43-5f26-4b32-b71d-5164569c4acb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:57.303017 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6663da43-5f26-4b32-b71d-5164569c4acb service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:57.312878 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1100/2184] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:44:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:44:57.956210 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dbda2a-6f10-4fcc-8e6f-4bb8f0d95ff8 service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:44:57.969332 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dbda2a-6f10-4fcc-8e6f-4bb8f0d95ff8 service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:44:57.974419 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31727e92-bf1c-49dd-81bd-5be3ab4b94af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:44:57.975556 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31727e92-bf1c-49dd-81bd-5be3ab4b94af service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:44:57.976495 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31727e92-bf1c-49dd-81bd-5be3ab4b94af service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:44:57.976495 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31727e92-bf1c-49dd-81bd-5be3ab4b94af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:44:57.980603 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-31727e92-bf1c-49dd-81bd-5be3ab4b94af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:44:58.072942 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31727e92-bf1c-49dd-81bd-5be3ab4b94af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:44:58.074432 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31727e92-bf1c-49dd-81bd-5be3ab4b94af service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:44:58.084560 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1085/2185] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:44:57 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:45:07.308201 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=15 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:45:08.028977 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6663da43-5f26-4b32-b71d-5164569c4acb service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:45:08.080251 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:45:08.091940 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6663da43-5f26-4b32-b71d-5164569c4acb service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:45:08.097088 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d677b204-1b9b-4c02-93ea-03e0aef1706e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:45:08.097991 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677b204-1b9b-4c02-93ea-03e0aef1706e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:45:08.098294 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677b204-1b9b-4c02-93ea-03e0aef1706e admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:45:08.098555 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677b204-1b9b-4c02-93ea-03e0aef1706e admin admin] PATH_INFO: `/v3/users/be97e1eab35e4245958d03a9a4f3d429` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:45:08.099838 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d677b204-1b9b-4c02-93ea-03e0aef1706e admin admin] RBAC: Authorizing `identity:delete_user(user_id=be97e1eab35e4245958d03a9a4f3d429)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:45:08.101752 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d677b204-1b9b-4c02-93ea-03e0aef1706e admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:45:08.153745 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-d677b204-1b9b-4c02-93ea-03e0aef1706e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be97e1eab35e4245958d03a9a4f3d429'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:08.154415 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-d677b204-1b9b-4c02-93ea-03e0aef1706e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be97e1eab35e4245958d03a9a4f3d429'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:08.154992 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-d677b204-1b9b-4c02-93ea-03e0aef1706e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be97e1eab35e4245958d03a9a4f3d429'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:08.164101 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-d677b204-1b9b-4c02-93ea-03e0aef1706e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be97e1eab35e4245958d03a9a4f3d429'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:08.178203 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-d677b204-1b9b-4c02-93ea-03e0aef1706e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be97e1eab35e4245958d03a9a4f3d429'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:08.191245 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1101/2186] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:45:08 2022] DELETE /identity/v3/users/be97e1eab35e4245958d03a9a4f3d429 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:45:08.196776 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fdb6fb-e9c0-4344-9e06-84d1a6d6842c None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:45:08.197070 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fdb6fb-e9c0-4344-9e06-84d1a6d6842c None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:45:08.197386 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fdb6fb-e9c0-4344-9e06-84d1a6d6842c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:45:08.215443 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-70fdb6fb-e9c0-4344-9e06-84d1a6d6842c None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:45:08.216592 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-70fdb6fb-e9c0-4344-9e06-84d1a6d6842c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:45:08.297978 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d677b204-1b9b-4c02-93ea-03e0aef1706e admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:45:08.298083 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1086/2187] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:45:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:45:08.299523 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d677b204-1b9b-4c02-93ea-03e0aef1706e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:45:08.309918 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fdb6fb-e9c0-4344-9e06-84d1a6d6842c None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:45:08.311631 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-70fdb6fb-e9c0-4344-9e06-84d1a6d6842c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:45:08.313200 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d677b204-1b9b-4c02-93ea-03e0aef1706e admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:45:08.318409 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c946de14-5e5d-46b1-b8ed-7e56fcac7212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:45:08.319368 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c946de14-5e5d-46b1-b8ed-7e56fcac7212 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:45:08.319727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c946de14-5e5d-46b1-b8ed-7e56fcac7212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:45:08.320077 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c946de14-5e5d-46b1-b8ed-7e56fcac7212 admin admin] PATH_INFO: `/v3/users/fc94434a2665492c80ce156f599b64cc` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:45:08.325930 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c946de14-5e5d-46b1-b8ed-7e56fcac7212 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc94434a2665492c80ce156f599b64cc)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:45:08.328439 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c946de14-5e5d-46b1-b8ed-7e56fcac7212 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:45:08.372877 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fdb6fb-e9c0-4344-9e06-84d1a6d6842c None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:45:08.376667 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be22866-64f6-4e39-822d-eae37e4debeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:45:08.377372 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be22866-64f6-4e39-822d-eae37e4debeb service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:45:08.377665 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be22866-64f6-4e39-822d-eae37e4debeb service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:45:08.377947 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be22866-64f6-4e39-822d-eae37e4debeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:45:08.382324 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-c946de14-5e5d-46b1-b8ed-7e56fcac7212 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc94434a2665492c80ce156f599b64cc'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:08.382978 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-c946de14-5e5d-46b1-b8ed-7e56fcac7212 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc94434a2665492c80ce156f599b64cc'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:08.383594 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-c946de14-5e5d-46b1-b8ed-7e56fcac7212 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc94434a2665492c80ce156f599b64cc'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:08.387510 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be22866-64f6-4e39-822d-eae37e4debeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:45:08.388955 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be22866-64f6-4e39-822d-eae37e4debeb service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:45:08.391983 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-c946de14-5e5d-46b1-b8ed-7e56fcac7212 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc94434a2665492c80ce156f599b64cc'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:08.399798 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1087/2188] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:45:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 16:45:08.408769 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-c946de14-5e5d-46b1-b8ed-7e56fcac7212 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc94434a2665492c80ce156f599b64cc'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:08.410766 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:45:08.424426 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1102/2189] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:45:08 2022] DELETE /identity/v3/users/fc94434a2665492c80ce156f599b64cc => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:45:08.429082 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f7b19-5aea-4d66-88fa-dbe4596c9f51 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:45:08.429935 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f7b19-5aea-4d66-88fa-dbe4596c9f51 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:45:08.430310 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f7b19-5aea-4d66-88fa-dbe4596c9f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:45:08.449252 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-216f7b19-5aea-4d66-88fa-dbe4596c9f51 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:45:08.450096 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-216f7b19-5aea-4d66-88fa-dbe4596c9f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:45:08.524095 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1088/2190] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:45:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:45:08.534412 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c946de14-5e5d-46b1-b8ed-7e56fcac7212 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:45:08.535834 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-c946de14-5e5d-46b1-b8ed-7e56fcac7212 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:45:08.602948 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c946de14-5e5d-46b1-b8ed-7e56fcac7212 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:45:08.607686 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c136d051-5c70-43aa-ab05-511c8c44aedc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:45:08.608121 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136d051-5c70-43aa-ab05-511c8c44aedc service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:45:08.608417 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136d051-5c70-43aa-ab05-511c8c44aedc service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:45:08.609051 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136d051-5c70-43aa-ab05-511c8c44aedc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:45:08.621440 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c136d051-5c70-43aa-ab05-511c8c44aedc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:45:08.623388 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c136d051-5c70-43aa-ab05-511c8c44aedc service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:45:08.637740 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1103/2191] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:45:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 16:45:08.855450 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f7b19-5aea-4d66-88fa-dbe4596c9f51 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:45:08.856958 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-216f7b19-5aea-4d66-88fa-dbe4596c9f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:45:08.868714 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f7b19-5aea-4d66-88fa-dbe4596c9f51 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:45:08.873455 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcab12d-12c4-4198-9dd4-6d36cf3d0300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:45:08.874916 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcab12d-12c4-4198-9dd4-6d36cf3d0300 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:45:08.875230 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcab12d-12c4-4198-9dd4-6d36cf3d0300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:45:08.875475 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcab12d-12c4-4198-9dd4-6d36cf3d0300 admin admin] PATH_INFO: `/v3/projects/47516a554956442a9cf0330aa3f5bb14` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:45:08.876590 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcab12d-12c4-4198-9dd4-6d36cf3d0300 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47516a554956442a9cf0330aa3f5bb14)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:45:08.879712 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcab12d-12c4-4198-9dd4-6d36cf3d0300 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:45:08.980592 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:45:08.989415 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-4bcab12d-12c4-4198-9dd4-6d36cf3d0300 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47516a554956442a9cf0330aa3f5bb14', 'request_id': 'req-4bcab12d-12c4-4198-9dd4-6d36cf3d0300'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:08.995454 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-4bcab12d-12c4-4198-9dd4-6d36cf3d0300 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47516a554956442a9cf0330aa3f5bb14', 'request_id': 'req-4bcab12d-12c4-4198-9dd4-6d36cf3d0300'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:09.002664 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-4bcab12d-12c4-4198-9dd4-6d36cf3d0300 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47516a554956442a9cf0330aa3f5bb14', 'request_id': 'req-4bcab12d-12c4-4198-9dd4-6d36cf3d0300'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:09.021789 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-4bcab12d-12c4-4198-9dd4-6d36cf3d0300 admin admin] The token cache is being invalidate because project 47516a554956442a9cf0330aa3f5bb14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:45:09.022080 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-4bcab12d-12c4-4198-9dd4-6d36cf3d0300 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:09.022891 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-4bcab12d-12c4-4198-9dd4-6d36cf3d0300 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:09.024734 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1089/2192] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:45:08 2022] DELETE /identity/v3/projects/47516a554956442a9cf0330aa3f5bb14 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:45:09.057403 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c136d051-5c70-43aa-ab05-511c8c44aedc service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:45:09.059576 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-c136d051-5c70-43aa-ab05-511c8c44aedc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:45:09.078608 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c136d051-5c70-43aa-ab05-511c8c44aedc service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:45:09.151700 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bd2f9a-0e70-43cd-9ea1-533fc9ee9ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:45:09.152603 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd2f9a-0e70-43cd-9ea1-533fc9ee9ea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:45:09.152895 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd2f9a-0e70-43cd-9ea1-533fc9ee9ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:45:09.153213 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd2f9a-0e70-43cd-9ea1-533fc9ee9ea6 admin admin] PATH_INFO: `/v3/projects/d4e27cee4a8745bbad913909b6f7dd4a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:45:09.154890 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bd2f9a-0e70-43cd-9ea1-533fc9ee9ea6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4e27cee4a8745bbad913909b6f7dd4a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:45:09.156888 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bd2f9a-0e70-43cd-9ea1-533fc9ee9ea6 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:45:09.211585 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-77bd2f9a-0e70-43cd-9ea1-533fc9ee9ea6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4e27cee4a8745bbad913909b6f7dd4a', 'request_id': 'req-77bd2f9a-0e70-43cd-9ea1-533fc9ee9ea6'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:09.218562 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-77bd2f9a-0e70-43cd-9ea1-533fc9ee9ea6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4e27cee4a8745bbad913909b6f7dd4a', 'request_id': 'req-77bd2f9a-0e70-43cd-9ea1-533fc9ee9ea6'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:09.230115 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-77bd2f9a-0e70-43cd-9ea1-533fc9ee9ea6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4e27cee4a8745bbad913909b6f7dd4a', 'request_id': 'req-77bd2f9a-0e70-43cd-9ea1-533fc9ee9ea6'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:09.245735 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-77bd2f9a-0e70-43cd-9ea1-533fc9ee9ea6 admin admin] The token cache is being invalidate because project d4e27cee4a8745bbad913909b6f7dd4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:45:09.246054 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-77bd2f9a-0e70-43cd-9ea1-533fc9ee9ea6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:09.246707 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-77bd2f9a-0e70-43cd-9ea1-533fc9ee9ea6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:09.248264 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1104/2193] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:45:09 2022] DELETE /identity/v3/projects/d4e27cee4a8745bbad913909b6f7dd4a => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:45:09.252917 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcab12d-12c4-4198-9dd4-6d36cf3d0300 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:45:09.254645 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-4bcab12d-12c4-4198-9dd4-6d36cf3d0300 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:45:09.267148 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcab12d-12c4-4198-9dd4-6d36cf3d0300 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:45:09.332060 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f00de4-ea98-47b5-b202-508568d1c756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:45:09.332868 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f00de4-ea98-47b5-b202-508568d1c756 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:45:09.333180 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f00de4-ea98-47b5-b202-508568d1c756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:45:09.333457 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f00de4-ea98-47b5-b202-508568d1c756 admin admin] PATH_INFO: `/v3/users/818f0dfcd232420d921088e932b7d3d7` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:45:09.334716 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f00de4-ea98-47b5-b202-508568d1c756 admin admin] RBAC: Authorizing `identity:delete_user(user_id=818f0dfcd232420d921088e932b7d3d7)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:45:09.336636 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f00de4-ea98-47b5-b202-508568d1c756 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:45:09.389165 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-14f00de4-ea98-47b5-b202-508568d1c756 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '818f0dfcd232420d921088e932b7d3d7'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:09.390698 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-14f00de4-ea98-47b5-b202-508568d1c756 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '818f0dfcd232420d921088e932b7d3d7'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:09.402209 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-14f00de4-ea98-47b5-b202-508568d1c756 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '818f0dfcd232420d921088e932b7d3d7'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:09.416495 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-14f00de4-ea98-47b5-b202-508568d1c756 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '818f0dfcd232420d921088e932b7d3d7'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:09.417266 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-14f00de4-ea98-47b5-b202-508568d1c756 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '818f0dfcd232420d921088e932b7d3d7'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:09.431020 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1090/2194] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:45:09 2022] DELETE /identity/v3/users/818f0dfcd232420d921088e932b7d3d7 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:45:09.785436 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bd2f9a-0e70-43cd-9ea1-533fc9ee9ea6 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:45:09.787398 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-77bd2f9a-0e70-43cd-9ea1-533fc9ee9ea6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:45:09.801492 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bd2f9a-0e70-43cd-9ea1-533fc9ee9ea6 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:45:09.857444 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:45:09.905856 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376db0f2-a206-4a9e-b353-6aac258a7628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:45:09.906718 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376db0f2-a206-4a9e-b353-6aac258a7628 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:45:09.907074 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376db0f2-a206-4a9e-b353-6aac258a7628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:45:09.907514 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376db0f2-a206-4a9e-b353-6aac258a7628 admin admin] PATH_INFO: `/v3/projects/9e4278f8faf948068f633e00c8826a52` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:45:09.908808 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376db0f2-a206-4a9e-b353-6aac258a7628 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e4278f8faf948068f633e00c8826a52)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:45:09.911046 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376db0f2-a206-4a9e-b353-6aac258a7628 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:45:09.984366 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-376db0f2-a206-4a9e-b353-6aac258a7628 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e4278f8faf948068f633e00c8826a52', 'request_id': 'req-376db0f2-a206-4a9e-b353-6aac258a7628'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:09.997932 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-376db0f2-a206-4a9e-b353-6aac258a7628 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e4278f8faf948068f633e00c8826a52', 'request_id': 'req-376db0f2-a206-4a9e-b353-6aac258a7628'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:10.005649 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-376db0f2-a206-4a9e-b353-6aac258a7628 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e4278f8faf948068f633e00c8826a52', 'request_id': 'req-376db0f2-a206-4a9e-b353-6aac258a7628'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:10.024464 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-376db0f2-a206-4a9e-b353-6aac258a7628 admin admin] The token cache is being invalidate because project 9e4278f8faf948068f633e00c8826a52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:45:10.024759 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-376db0f2-a206-4a9e-b353-6aac258a7628 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:10.025427 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-376db0f2-a206-4a9e-b353-6aac258a7628 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:10.027190 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1105/2195] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:45:09 2022] DELETE /identity/v3/projects/9e4278f8faf948068f633e00c8826a52 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:45:10.031398 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f00de4-ea98-47b5-b202-508568d1c756 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:45:10.032864 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-14f00de4-ea98-47b5-b202-508568d1c756 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:45:10.047545 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f00de4-ea98-47b5-b202-508568d1c756 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:45:10.095569 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:45:10.112559 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea78c2c-f099-45db-83e6-c42e352c595b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:45:10.113509 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea78c2c-f099-45db-83e6-c42e352c595b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:45:10.113794 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea78c2c-f099-45db-83e6-c42e352c595b admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:45:10.114073 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea78c2c-f099-45db-83e6-c42e352c595b admin admin] PATH_INFO: `/v3/users/6c655f01b28a46d692b1ff9fa79b4e07` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:45:10.115290 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea78c2c-f099-45db-83e6-c42e352c595b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c655f01b28a46d692b1ff9fa79b4e07)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:45:10.117294 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea78c2c-f099-45db-83e6-c42e352c595b admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:45:10.190058 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0ea78c2c-f099-45db-83e6-c42e352c595b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c655f01b28a46d692b1ff9fa79b4e07'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:10.190730 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0ea78c2c-f099-45db-83e6-c42e352c595b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c655f01b28a46d692b1ff9fa79b4e07'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:10.200063 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0ea78c2c-f099-45db-83e6-c42e352c595b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c655f01b28a46d692b1ff9fa79b4e07'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:10.217110 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0ea78c2c-f099-45db-83e6-c42e352c595b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c655f01b28a46d692b1ff9fa79b4e07'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:10.218432 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0ea78c2c-f099-45db-83e6-c42e352c595b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c655f01b28a46d692b1ff9fa79b4e07'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:10.232186 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1091/2196] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:45:10 2022] DELETE /identity/v3/users/6c655f01b28a46d692b1ff9fa79b4e07 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:45:10.716169 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376db0f2-a206-4a9e-b353-6aac258a7628 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:45:10.717567 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-376db0f2-a206-4a9e-b353-6aac258a7628 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:45:10.732563 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376db0f2-a206-4a9e-b353-6aac258a7628 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:45:10.797421 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbe5a8d-15f8-4671-ad90-684dd7ccc1e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:45:10.798299 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe5a8d-15f8-4671-ad90-684dd7ccc1e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:45:10.798589 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe5a8d-15f8-4671-ad90-684dd7ccc1e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:45:10.798870 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe5a8d-15f8-4671-ad90-684dd7ccc1e7 admin admin] PATH_INFO: `/v3/projects/061467f24c364080a76433e2f48e6077` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:45:10.800155 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbe5a8d-15f8-4671-ad90-684dd7ccc1e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=061467f24c364080a76433e2f48e6077)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:45:10.802136 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbe5a8d-15f8-4671-ad90-684dd7ccc1e7 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:45:10.849261 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6dbe5a8d-15f8-4671-ad90-684dd7ccc1e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '061467f24c364080a76433e2f48e6077', 'request_id': 'req-6dbe5a8d-15f8-4671-ad90-684dd7ccc1e7'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:10.853893 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:45:10.855275 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6dbe5a8d-15f8-4671-ad90-684dd7ccc1e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '061467f24c364080a76433e2f48e6077', 'request_id': 'req-6dbe5a8d-15f8-4671-ad90-684dd7ccc1e7'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:10.863514 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6dbe5a8d-15f8-4671-ad90-684dd7ccc1e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '061467f24c364080a76433e2f48e6077', 'request_id': 'req-6dbe5a8d-15f8-4671-ad90-684dd7ccc1e7'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:10.876421 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6dbe5a8d-15f8-4671-ad90-684dd7ccc1e7 admin admin] The token cache is being invalidate because project 061467f24c364080a76433e2f48e6077 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:45:10.876731 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6dbe5a8d-15f8-4671-ad90-684dd7ccc1e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:10.877383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6dbe5a8d-15f8-4671-ad90-684dd7ccc1e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:10.879124 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1106/2197] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:45:10 2022] DELETE /identity/v3/projects/061467f24c364080a76433e2f48e6077 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:45:10.885181 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b5181-2832-4d1c-807f-7cc3374d3d42 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:45:10.885638 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b5181-2832-4d1c-807f-7cc3374d3d42 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:45:10.885797 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b5181-2832-4d1c-807f-7cc3374d3d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:45:10.907038 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-7d3b5181-2832-4d1c-807f-7cc3374d3d42 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:45:10.908163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-7d3b5181-2832-4d1c-807f-7cc3374d3d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:45:10.972875 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1092/2198] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:45:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:45:10.980777 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c835e-17af-489c-9637-1a0379f114d6 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:45:10.981230 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c835e-17af-489c-9637-1a0379f114d6 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:45:10.981420 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c835e-17af-489c-9637-1a0379f114d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:45:11.018275 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-bb8c835e-17af-489c-9637-1a0379f114d6 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:45:11.020049 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-bb8c835e-17af-489c-9637-1a0379f114d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:45:11.034998 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1107/2199] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:45:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:45:11.040351 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3b5181-2832-4d1c-807f-7cc3374d3d42 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:45:11.052143 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3b5181-2832-4d1c-807f-7cc3374d3d42 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:45:11.059424 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ed4a5-790e-4dac-94cb-d611bdef8f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:45:11.060158 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ed4a5-790e-4dac-94cb-d611bdef8f5d admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:45:11.060665 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ed4a5-790e-4dac-94cb-d611bdef8f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:45:11.060952 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ed4a5-790e-4dac-94cb-d611bdef8f5d admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:45:11.062296 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007ed4a5-790e-4dac-94cb-d611bdef8f5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:45:11.062617 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007ed4a5-790e-4dac-94cb-d611bdef8f5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:45:11.064548 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007ed4a5-790e-4dac-94cb-d611bdef8f5d admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:45:11.085574 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1093/2200] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:45:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:45:11.087430 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8c835e-17af-489c-9637-1a0379f114d6 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:45:11.108864 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8c835e-17af-489c-9637-1a0379f114d6 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:45:11.114835 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c162a4-6753-4b71-a2ab-5c248c835739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:45:11.115712 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c162a4-6753-4b71-a2ab-5c248c835739 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:45:11.116005 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c162a4-6753-4b71-a2ab-5c248c835739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:45:11.116285 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c162a4-6753-4b71-a2ab-5c248c835739 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:45:11.117015 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c162a4-6753-4b71-a2ab-5c248c835739 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:45:11.119039 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c162a4-6753-4b71-a2ab-5c248c835739 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:45:11.149530 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1108/2201] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:45:11 2022] POST /identity/v3/projects => generated 373 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:45:11.153747 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ed4a5-790e-4dac-94cb-d611bdef8f5d admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:45:11.165071 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ed4a5-790e-4dac-94cb-d611bdef8f5d admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:45:11.227845 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:45:11.238774 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96726c9-7666-444e-bddf-5d4522f43350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:45:11.239476 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96726c9-7666-444e-bddf-5d4522f43350 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:45:11.239788 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96726c9-7666-444e-bddf-5d4522f43350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:45:11.240100 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96726c9-7666-444e-bddf-5d4522f43350 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:45:11.240810 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96726c9-7666-444e-bddf-5d4522f43350 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:45:11.242849 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96726c9-7666-444e-bddf-5d4522f43350 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:45:11.263532 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1094/2202] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:45:11 2022] POST /identity/v3/users => generated 287 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:45:11.267721 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c162a4-6753-4b71-a2ab-5c248c835739 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:45:11.277941 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c162a4-6753-4b71-a2ab-5c248c835739 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:45:11.282636 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f8ae7-0d7e-4d6d-b526-fb70d9c5bfc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:45:11.283289 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f8ae7-0d7e-4d6d-b526-fb70d9c5bfc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:45:11.283576 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f8ae7-0d7e-4d6d-b526-fb70d9c5bfc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:45:11.283855 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f8ae7-0d7e-4d6d-b526-fb70d9c5bfc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:45:11.284589 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5f8ae7-0d7e-4d6d-b526-fb70d9c5bfc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:45:11.286236 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5f8ae7-0d7e-4d6d-b526-fb70d9c5bfc4 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:45:11.301924 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1109/2203] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:45:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:45:11.303329 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96726c9-7666-444e-bddf-5d4522f43350 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:45:11.314260 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96726c9-7666-444e-bddf-5d4522f43350 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:45:11.318965 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b2d7f0-a987-459c-9c26-ad516c97ec88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:45:11.319627 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b2d7f0-a987-459c-9c26-ad516c97ec88 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:45:11.319942 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b2d7f0-a987-459c-9c26-ad516c97ec88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:45:11.320273 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b2d7f0-a987-459c-9c26-ad516c97ec88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:45:11.320944 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b2d7f0-a987-459c-9c26-ad516c97ec88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:45:11.322600 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b2d7f0-a987-459c-9c26-ad516c97ec88 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:45:11.332818 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1095/2204] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:45:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:45:11.337266 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f8ae7-0d7e-4d6d-b526-fb70d9c5bfc4 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:45:11.347856 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f8ae7-0d7e-4d6d-b526-fb70d9c5bfc4 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:45:11.352490 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a4a732-06c6-4489-8348-819f685d4f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:45:11.353395 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a4a732-06c6-4489-8348-819f685d4f3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:45:11.353684 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a4a732-06c6-4489-8348-819f685d4f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:45:11.353963 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a4a732-06c6-4489-8348-819f685d4f3b admin admin] PATH_INFO: `/v3/projects/6c7652596e3b41d285e874ca1a675677/users/1d0fe4ce272e46509e8b744e17664ccd/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:45:11.369337 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a4a732-06c6-4489-8348-819f685d4f3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c7652596e3b41d285e874ca1a675677, user_id=1d0fe4ce272e46509e8b744e17664ccd, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:45:11.372009 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a4a732-06c6-4489-8348-819f685d4f3b admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:45:11.386693 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1110/2205] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:45:11 2022] PUT /identity/v3/projects/6c7652596e3b41d285e874ca1a675677/users/1d0fe4ce272e46509e8b744e17664ccd/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:45:11.397029 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2916e539-1b71-402f-aeb3-42a6b6a54e60 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:45:11.397404 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2916e539-1b71-402f-aeb3-42a6b6a54e60 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:45:11.397692 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2916e539-1b71-402f-aeb3-42a6b6a54e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:45:11.424329 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-2916e539-1b71-402f-aeb3-42a6b6a54e60 None None] MFA Rules not processed for user `1d0fe4ce272e46509e8b744e17664ccd`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:45:11.425495 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-2916e539-1b71-402f-aeb3-42a6b6a54e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:45:11.488050 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1096/2206] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:45:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:45:11.498690 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a4a732-06c6-4489-8348-819f685d4f3b admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:45:11.500967 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-25a4a732-06c6-4489-8348-819f685d4f3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:45:11.580850 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a4a732-06c6-4489-8348-819f685d4f3b admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:45:11.584656 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1a9cd4-f329-4d23-919e-1a3212d490dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:45:11.585370 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a9cd4-f329-4d23-919e-1a3212d490dc service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:45:11.585659 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a9cd4-f329-4d23-919e-1a3212d490dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:45:11.585939 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a9cd4-f329-4d23-919e-1a3212d490dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:45:11.594954 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1a9cd4-f329-4d23-919e-1a3212d490dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:45:11.596390 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1a9cd4-f329-4d23-919e-1a3212d490dc service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:45:11.605038 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1111/2207] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:45:11 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:45:12.950201 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca27b61-3c65-4c55-b683-a305a675477d None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:45:12.950707 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca27b61-3c65-4c55-b683-a305a675477d None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:45:12.950799 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca27b61-3c65-4c55-b683-a305a675477d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:45:12.968229 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-0ca27b61-3c65-4c55-b683-a305a675477d None None] MFA Rules not processed for user `1d0fe4ce272e46509e8b744e17664ccd`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:45:12.969478 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-0ca27b61-3c65-4c55-b683-a305a675477d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:45:12.976309 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1097/2208] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:45:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:45:19.190594 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b701c9-f60b-4302-af18-c9f764df6b41 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:45:19.191125 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b701c9-f60b-4302-af18-c9f764df6b41 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:45:19.191219 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b701c9-f60b-4302-af18-c9f764df6b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:45:19.209627 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-02b701c9-f60b-4302-af18-c9f764df6b41 None None] MFA Rules not processed for user `6e88db1bf9294decaf171756772a5066`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:45:19.210880 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-02b701c9-f60b-4302-af18-c9f764df6b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:45:19.273587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1112/2209] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:45:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:45:22.962904 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=45,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:45:29.265080 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:45:44.443662 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b3fd8-7a75-4873-8946-ae1573125452 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:45:44.444214 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b3fd8-7a75-4873-8946-ae1573125452 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:45:44.444660 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b3fd8-7a75-4873-8946-ae1573125452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:45:44.506586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-773b3fd8-7a75-4873-8946-ae1573125452 None None] MFA Rules not processed for user `a30dc1dd2b00463bb3c7957864c56043`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:45:44.509109 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-773b3fd8-7a75-4873-8946-ae1573125452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:45:44.602314 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1098/2210] 10.222.0.36 () {58 vars in 1009 bytes} [Thu May 19 16:45:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:45:44.652935 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b701c9-f60b-4302-af18-c9f764df6b41 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:45:44.654490 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-02b701c9-f60b-4302-af18-c9f764df6b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:45:44.764071 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b701c9-f60b-4302-af18-c9f764df6b41 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:45:44.769947 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77232a60-ff8d-4ca6-9519-dbb2f5964d37 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffe7cebc6f3b40d8a8e1866ed2da9822', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:45:44.770607 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77232a60-ff8d-4ca6-9519-dbb2f5964d37 service swift] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:45:44.770896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77232a60-ff8d-4ca6-9519-dbb2f5964d37 service swift] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:45:44.771175 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77232a60-ff8d-4ca6-9519-dbb2f5964d37 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:45:44.779872 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77232a60-ff8d-4ca6-9519-dbb2f5964d37 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:45:44.781537 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77232a60-ff8d-4ca6-9519-dbb2f5964d37 service swift] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:45:44.790748 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1113/2211] 10.222.0.36 () {60 vars in 1384 bytes} [Thu May 19 16:45:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 16:45:45.394422 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69123ae-dbfc-40c3-9ba3-a758cc56b76a None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:45:45.394683 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69123ae-dbfc-40c3-9ba3-a758cc56b76a None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:45:45.395000 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69123ae-dbfc-40c3-9ba3-a758cc56b76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:45:45.415516 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-d69123ae-dbfc-40c3-9ba3-a758cc56b76a None None] MFA Rules not processed for user `a30dc1dd2b00463bb3c7957864c56043`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:45:45.416653 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-d69123ae-dbfc-40c3-9ba3-a758cc56b76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:45:45.424629 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1099/2212] 10.222.0.36 () {58 vars in 967 bytes} [Thu May 19 16:45:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:45:45.441694 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77232a60-ff8d-4ca6-9519-dbb2f5964d37 service swift] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:45:45.452737 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77232a60-ff8d-4ca6-9519-dbb2f5964d37 service swift] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:45:45.457938 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c720b60-949e-4ffb-9657-4bfd319c167a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffe7cebc6f3b40d8a8e1866ed2da9822', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:45:45.458580 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c720b60-949e-4ffb-9657-4bfd319c167a service swift] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:45:45.458861 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c720b60-949e-4ffb-9657-4bfd319c167a service swift] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:45:45.459138 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c720b60-949e-4ffb-9657-4bfd319c167a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:45:45.467786 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c720b60-949e-4ffb-9657-4bfd319c167a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:45:45.469423 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c720b60-949e-4ffb-9657-4bfd319c167a service swift] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:45:45.478471 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1114/2213] 10.222.0.36 () {60 vars in 1384 bytes} [Thu May 19 16:45:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 16:45:45.690405 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e52f6d-3f3c-4a2f-9292-5a0d49dee062 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:45:45.690936 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e52f6d-3f3c-4a2f-9292-5a0d49dee062 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:45:45.691024 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e52f6d-3f3c-4a2f-9292-5a0d49dee062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:45:45.710647 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-c3e52f6d-3f3c-4a2f-9292-5a0d49dee062 None None] MFA Rules not processed for user `a30dc1dd2b00463bb3c7957864c56043`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:45:45.711896 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-c3e52f6d-3f3c-4a2f-9292-5a0d49dee062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:45:45.719309 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1100/2214] 10.222.0.36 () {58 vars in 1009 bytes} [Thu May 19 16:45:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:45:45.733037 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c720b60-949e-4ffb-9657-4bfd319c167a service swift] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:45:45.745617 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c720b60-949e-4ffb-9657-4bfd319c167a service swift] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:45:45.751594 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a20c803-733c-4e72-be08-9d9da8b5f390 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffe7cebc6f3b40d8a8e1866ed2da9822', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:45:45.752257 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a20c803-733c-4e72-be08-9d9da8b5f390 service swift] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:45:45.752544 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a20c803-733c-4e72-be08-9d9da8b5f390 service swift] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:45:45.752820 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a20c803-733c-4e72-be08-9d9da8b5f390 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:45:45.762730 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a20c803-733c-4e72-be08-9d9da8b5f390 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:45:45.764506 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a20c803-733c-4e72-be08-9d9da8b5f390 service swift] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:45:45.773330 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1115/2215] 10.222.0.36 () {60 vars in 1384 bytes} [Thu May 19 16:45:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 16:45:46.152437 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a65ca6-78c2-46fa-9d29-df946561e0ae None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:45:46.152728 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a65ca6-78c2-46fa-9d29-df946561e0ae None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:45:46.153036 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a65ca6-78c2-46fa-9d29-df946561e0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:45:46.171425 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-b1a65ca6-78c2-46fa-9d29-df946561e0ae None None] MFA Rules not processed for user `a30dc1dd2b00463bb3c7957864c56043`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:45:46.172644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-b1a65ca6-78c2-46fa-9d29-df946561e0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:45:46.180499 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1101/2216] 10.222.0.36 () {58 vars in 967 bytes} [Thu May 19 16:45:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:45:46.194448 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a20c803-733c-4e72-be08-9d9da8b5f390 service swift] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:45:46.205701 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a20c803-733c-4e72-be08-9d9da8b5f390 service swift] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:45:46.210856 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3002bd-aef4-4cbc-b03d-84fc182ce655 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffe7cebc6f3b40d8a8e1866ed2da9822', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:45:46.211502 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3002bd-aef4-4cbc-b03d-84fc182ce655 service swift] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:45:46.211783 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3002bd-aef4-4cbc-b03d-84fc182ce655 service swift] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:45:46.212064 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3002bd-aef4-4cbc-b03d-84fc182ce655 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:45:46.220940 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3002bd-aef4-4cbc-b03d-84fc182ce655 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:45:46.222618 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3002bd-aef4-4cbc-b03d-84fc182ce655 service swift] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:45:46.231520 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1116/2217] 10.222.0.36 () {60 vars in 1384 bytes} [Thu May 19 16:45:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 16:45:51.538491 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a65ca6-78c2-46fa-9d29-df946561e0ae None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:45:51.622962 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a65ca6-78c2-46fa-9d29-df946561e0ae None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:45:51.629184 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7edf552-0bb0-4044-a8d3-c25a78a5473f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:45:51.630108 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7edf552-0bb0-4044-a8d3-c25a78a5473f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:45:51.630398 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7edf552-0bb0-4044-a8d3-c25a78a5473f admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:45:51.630677 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7edf552-0bb0-4044-a8d3-c25a78a5473f admin admin] PATH_INFO: `/v3/users/1d0fe4ce272e46509e8b744e17664ccd` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:45:51.631896 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7edf552-0bb0-4044-a8d3-c25a78a5473f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d0fe4ce272e46509e8b744e17664ccd)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:45:51.645276 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7edf552-0bb0-4044-a8d3-c25a78a5473f admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:45:51.687423 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:45:51.695616 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b7edf552-0bb0-4044-a8d3-c25a78a5473f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d0fe4ce272e46509e8b744e17664ccd'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:51.696878 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b7edf552-0bb0-4044-a8d3-c25a78a5473f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d0fe4ce272e46509e8b744e17664ccd'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:51.707389 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b7edf552-0bb0-4044-a8d3-c25a78a5473f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d0fe4ce272e46509e8b744e17664ccd'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:51.720081 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b7edf552-0bb0-4044-a8d3-c25a78a5473f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d0fe4ce272e46509e8b744e17664ccd'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:51.720822 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-b7edf552-0bb0-4044-a8d3-c25a78a5473f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d0fe4ce272e46509e8b744e17664ccd'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:51.733564 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1102/2218] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:45:51 2022] DELETE /identity/v3/users/1d0fe4ce272e46509e8b744e17664ccd => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:45:51.738158 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f855a-c4a1-45d7-af08-e290717dd892 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:45:51.738612 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f855a-c4a1-45d7-af08-e290717dd892 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:45:51.738860 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f855a-c4a1-45d7-af08-e290717dd892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:45:51.758214 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-5a3f855a-c4a1-45d7-af08-e290717dd892 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:45:51.759419 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-5a3f855a-c4a1-45d7-af08-e290717dd892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:45:51.828957 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1117/2219] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:45:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:45:51.840055 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7edf552-0bb0-4044-a8d3-c25a78a5473f admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:45:51.841572 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-b7edf552-0bb0-4044-a8d3-c25a78a5473f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:45:51.910165 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7edf552-0bb0-4044-a8d3-c25a78a5473f admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:45:51.913978 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68dfe88-67ee-4119-a3cc-fe9c45e7f2aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:45:51.914636 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68dfe88-67ee-4119-a3cc-fe9c45e7f2aa service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:45:51.915003 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68dfe88-67ee-4119-a3cc-fe9c45e7f2aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:45:51.915194 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68dfe88-67ee-4119-a3cc-fe9c45e7f2aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:45:51.925612 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68dfe88-67ee-4119-a3cc-fe9c45e7f2aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:45:51.927063 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68dfe88-67ee-4119-a3cc-fe9c45e7f2aa service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:45:51.938192 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1103/2220] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:45:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:45:52.120050 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f855a-c4a1-45d7-af08-e290717dd892 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:45:52.121526 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-5a3f855a-c4a1-45d7-af08-e290717dd892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:45:52.134526 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f855a-c4a1-45d7-af08-e290717dd892 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:45:52.139509 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80973f1-5f3a-4897-bcd9-c56d3997079c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:45:52.140397 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80973f1-5f3a-4897-bcd9-c56d3997079c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:45:52.140837 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80973f1-5f3a-4897-bcd9-c56d3997079c admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:45:52.141322 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80973f1-5f3a-4897-bcd9-c56d3997079c admin admin] PATH_INFO: `/v3/projects/6c7652596e3b41d285e874ca1a675677` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:45:52.143617 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80973f1-5f3a-4897-bcd9-c56d3997079c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c7652596e3b41d285e874ca1a675677)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:45:52.145776 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80973f1-5f3a-4897-bcd9-c56d3997079c admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:45:52.187450 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:45:52.207435 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-d80973f1-5f3a-4897-bcd9-c56d3997079c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c7652596e3b41d285e874ca1a675677', 'request_id': 'req-d80973f1-5f3a-4897-bcd9-c56d3997079c'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:52.213446 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-d80973f1-5f3a-4897-bcd9-c56d3997079c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c7652596e3b41d285e874ca1a675677', 'request_id': 'req-d80973f1-5f3a-4897-bcd9-c56d3997079c'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:52.221360 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-d80973f1-5f3a-4897-bcd9-c56d3997079c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c7652596e3b41d285e874ca1a675677', 'request_id': 'req-d80973f1-5f3a-4897-bcd9-c56d3997079c'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:52.238330 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-d80973f1-5f3a-4897-bcd9-c56d3997079c admin admin] The token cache is being invalidate because project 6c7652596e3b41d285e874ca1a675677 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:45:52.238642 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-d80973f1-5f3a-4897-bcd9-c56d3997079c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:52.239309 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-d80973f1-5f3a-4897-bcd9-c56d3997079c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:45:52.240873 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1118/2221] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:45:52 2022] DELETE /identity/v3/projects/6c7652596e3b41d285e874ca1a675677 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:46:01.932742 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:DELETE=5,keystone:SELECT=40,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:46:02.233023 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:DELETE=6,keystone:SELECT=14,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:46:04.785541 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68dfe88-67ee-4119-a3cc-fe9c45e7f2aa service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:46:04.787074 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-d68dfe88-67ee-4119-a3cc-fe9c45e7f2aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:46:04.888063 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68dfe88-67ee-4119-a3cc-fe9c45e7f2aa service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:46:04.956383 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b36263-92c5-4f1c-97e5-ca0a96f4aef8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e158e5c87f4954aeb297f42dabdc16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:46:04.957046 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b36263-92c5-4f1c-97e5-ca0a96f4aef8 service placement] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:46:04.957363 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b36263-92c5-4f1c-97e5-ca0a96f4aef8 service placement] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:46:04.957646 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b36263-92c5-4f1c-97e5-ca0a96f4aef8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:46:04.967362 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-b5b36263-92c5-4f1c-97e5-ca0a96f4aef8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:46:04.984954 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b36263-92c5-4f1c-97e5-ca0a96f4aef8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:46:04.984954 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b36263-92c5-4f1c-97e5-ca0a96f4aef8 service placement] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:46:04.994626 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1104/2222] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:46:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:46:11.899098 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863b142-c210-4d75-a76f-104324699380 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:46:11.899619 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863b142-c210-4d75-a76f-104324699380 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:46:11.899726 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863b142-c210-4d75-a76f-104324699380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:46:11.917904 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-6863b142-c210-4d75-a76f-104324699380 None None] MFA Rules not processed for user `e9c52d463394487d94e55aed8c44068a`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:46:11.919130 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-6863b142-c210-4d75-a76f-104324699380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:46:11.973941 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1119/2223] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:46:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:46:14.987933 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=67 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:46:21.966741 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=25 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:46:47.811648 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682c166-9dc3-43a2-ab8d-0a79cefbd4ad None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:46:47.812281 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682c166-9dc3-43a2-ab8d-0a79cefbd4ad None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:46:47.812281 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682c166-9dc3-43a2-ab8d-0a79cefbd4ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:46:47.831287 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-b682c166-9dc3-43a2-ab8d-0a79cefbd4ad None None] MFA Rules not processed for user `0695c868a2d844089fcda29ea3653d5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:46:47.833113 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-b682c166-9dc3-43a2-ab8d-0a79cefbd4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:46:47.898760 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1105/2224] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:46:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:46:57.888003 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=25 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:47:20.217287 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ecb4a4-87ff-4c6a-b89f-3100275ab88d None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:47:20.217851 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ecb4a4-87ff-4c6a-b89f-3100275ab88d None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:47:20.218009 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ecb4a4-87ff-4c6a-b89f-3100275ab88d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:47:20.235815 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-c4ecb4a4-87ff-4c6a-b89f-3100275ab88d None None] MFA Rules not processed for user `a30dc1dd2b00463bb3c7957864c56043`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:47:20.237020 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-c4ecb4a4-87ff-4c6a-b89f-3100275ab88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:47:20.293661 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1120/2225] 10.222.0.36 () {58 vars in 1009 bytes} [Thu May 19 16:47:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:47:20.344002 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682c166-9dc3-43a2-ab8d-0a79cefbd4ad None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:47:20.345479 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-b682c166-9dc3-43a2-ab8d-0a79cefbd4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:47:20.427157 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682c166-9dc3-43a2-ab8d-0a79cefbd4ad None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:47:20.432145 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63065751-0c3e-4681-81d4-581deaeeb709 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffe7cebc6f3b40d8a8e1866ed2da9822', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:47:20.432787 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63065751-0c3e-4681-81d4-581deaeeb709 service swift] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:47:20.433077 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63065751-0c3e-4681-81d4-581deaeeb709 service swift] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:47:20.433375 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63065751-0c3e-4681-81d4-581deaeeb709 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:47:20.441909 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63065751-0c3e-4681-81d4-581deaeeb709 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:47:20.444816 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63065751-0c3e-4681-81d4-581deaeeb709 service swift] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:47:20.453754 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1106/2226] 10.222.0.36 () {60 vars in 1384 bytes} [Thu May 19 16:47:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 16:47:21.556727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ecb4a4-87ff-4c6a-b89f-3100275ab88d None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:47:21.558393 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-c4ecb4a4-87ff-4c6a-b89f-3100275ab88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:47:21.655590 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ecb4a4-87ff-4c6a-b89f-3100275ab88d None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:47:21.662125 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671c413-d623-440a-993a-c8b3c1e552f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:47:21.663628 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671c413-d623-440a-993a-c8b3c1e552f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:47:21.664213 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671c413-d623-440a-993a-c8b3c1e552f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:47:21.664752 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671c413-d623-440a-993a-c8b3c1e552f6 admin admin] PATH_INFO: `/v3/users/e9c52d463394487d94e55aed8c44068a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:47:21.666154 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f671c413-d623-440a-993a-c8b3c1e552f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9c52d463394487d94e55aed8c44068a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:47:21.668207 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f671c413-d623-440a-993a-c8b3c1e552f6 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:47:21.758821 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-f671c413-d623-440a-993a-c8b3c1e552f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9c52d463394487d94e55aed8c44068a'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:47:21.759959 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-f671c413-d623-440a-993a-c8b3c1e552f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9c52d463394487d94e55aed8c44068a'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:47:21.760396 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-f671c413-d623-440a-993a-c8b3c1e552f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9c52d463394487d94e55aed8c44068a'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:47:21.771057 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-f671c413-d623-440a-993a-c8b3c1e552f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9c52d463394487d94e55aed8c44068a'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:47:21.784264 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-f671c413-d623-440a-993a-c8b3c1e552f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9c52d463394487d94e55aed8c44068a'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:47:21.799459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1121/2227] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:47:21 2022] DELETE /identity/v3/users/e9c52d463394487d94e55aed8c44068a => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:47:22.005667 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63065751-0c3e-4681-81d4-581deaeeb709 service swift] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:47:22.039943 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63065751-0c3e-4681-81d4-581deaeeb709 service swift] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:47:22.183789 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18521887-dc3e-4ed1-a033-33cd7384e930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:47:22.185076 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18521887-dc3e-4ed1-a033-33cd7384e930 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:47:22.185630 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18521887-dc3e-4ed1-a033-33cd7384e930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:47:22.186109 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18521887-dc3e-4ed1-a033-33cd7384e930 admin admin] PATH_INFO: `/v3/projects/3cee796edf9b42538358459363ea24ca` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:47:22.190518 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18521887-dc3e-4ed1-a033-33cd7384e930 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cee796edf9b42538358459363ea24ca)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:47:22.193692 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18521887-dc3e-4ed1-a033-33cd7384e930 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:47:22.280634 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-18521887-dc3e-4ed1-a033-33cd7384e930 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cee796edf9b42538358459363ea24ca', 'request_id': 'req-18521887-dc3e-4ed1-a033-33cd7384e930'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:47:22.287372 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-18521887-dc3e-4ed1-a033-33cd7384e930 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cee796edf9b42538358459363ea24ca', 'request_id': 'req-18521887-dc3e-4ed1-a033-33cd7384e930'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:47:22.294531 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-18521887-dc3e-4ed1-a033-33cd7384e930 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cee796edf9b42538358459363ea24ca', 'request_id': 'req-18521887-dc3e-4ed1-a033-33cd7384e930'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:47:22.313212 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-18521887-dc3e-4ed1-a033-33cd7384e930 admin admin] The token cache is being invalidate because project 3cee796edf9b42538358459363ea24ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:47:22.313610 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-18521887-dc3e-4ed1-a033-33cd7384e930 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:47:22.314395 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-18521887-dc3e-4ed1-a033-33cd7384e930 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:47:22.316084 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1107/2228] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:47:22 2022] DELETE /identity/v3/projects/3cee796edf9b42538358459363ea24ca => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:47:22.326976 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b2db8-61fa-46f6-985e-9c1f17dfd764 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:47:22.326976 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b2db8-61fa-46f6-985e-9c1f17dfd764 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:47:22.326976 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b2db8-61fa-46f6-985e-9c1f17dfd764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:47:22.348860 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-198b2db8-61fa-46f6-985e-9c1f17dfd764 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:47:22.350284 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-198b2db8-61fa-46f6-985e-9c1f17dfd764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:47:22.381669 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:47:22.442431 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1122/2229] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:47:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:47:22.450174 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25587e2-2fc6-4e0e-a525-fa5b5e60543a None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:47:22.450606 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25587e2-2fc6-4e0e-a525-fa5b5e60543a None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:47:22.450987 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25587e2-2fc6-4e0e-a525-fa5b5e60543a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:47:22.475758 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-f25587e2-2fc6-4e0e-a525-fa5b5e60543a None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:47:22.478666 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-f25587e2-2fc6-4e0e-a525-fa5b5e60543a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:47:22.489588 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1108/2230] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:47:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:47:22.493705 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b2db8-61fa-46f6-985e-9c1f17dfd764 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:47:22.505513 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b2db8-61fa-46f6-985e-9c1f17dfd764 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:47:22.513082 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e159c046-eff8-4fac-b5b2-951a37586d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:47:22.513803 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159c046-eff8-4fac-b5b2-951a37586d6a admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:47:22.514098 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159c046-eff8-4fac-b5b2-951a37586d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:47:22.514380 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159c046-eff8-4fac-b5b2-951a37586d6a admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:47:22.515128 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e159c046-eff8-4fac-b5b2-951a37586d6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:47:22.515443 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e159c046-eff8-4fac-b5b2-951a37586d6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:47:22.517085 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e159c046-eff8-4fac-b5b2-951a37586d6a admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:47:22.537403 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1123/2231] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:47:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:47:22.538258 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25587e2-2fc6-4e0e-a525-fa5b5e60543a None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:47:22.548601 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25587e2-2fc6-4e0e-a525-fa5b5e60543a None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:47:22.566384 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e718c74-a342-48e8-b6d0-6e17b56e6ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:47:22.567118 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e718c74-a342-48e8-b6d0-6e17b56e6ac6 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:47:22.567413 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e718c74-a342-48e8-b6d0-6e17b56e6ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:47:22.567694 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e718c74-a342-48e8-b6d0-6e17b56e6ac6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:47:22.568402 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e718c74-a342-48e8-b6d0-6e17b56e6ac6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:47:22.570379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e718c74-a342-48e8-b6d0-6e17b56e6ac6 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:47:22.582175 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:47:22.603392 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1109/2232] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:47:22 2022] POST /identity/v3/projects => generated 389 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:47:22.608151 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e159c046-eff8-4fac-b5b2-951a37586d6a admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:47:22.618767 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e159c046-eff8-4fac-b5b2-951a37586d6a admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:47:22.687218 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55f62e0-c95d-4af4-b7d2-c0a1bf59ddd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:47:22.687936 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f62e0-c95d-4af4-b7d2-c0a1bf59ddd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:47:22.688389 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f62e0-c95d-4af4-b7d2-c0a1bf59ddd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:47:22.688680 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f62e0-c95d-4af4-b7d2-c0a1bf59ddd6 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:47:22.689444 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55f62e0-c95d-4af4-b7d2-c0a1bf59ddd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:47:22.691463 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55f62e0-c95d-4af4-b7d2-c0a1bf59ddd6 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:47:22.718692 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1124/2233] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:47:22 2022] POST /identity/v3/users => generated 295 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:47:22.722640 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e718c74-a342-48e8-b6d0-6e17b56e6ac6 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:47:22.736216 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e718c74-a342-48e8-b6d0-6e17b56e6ac6 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:47:22.742251 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b4166-4fdd-497a-8de2-a3f4f6350a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:47:22.742940 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b4166-4fdd-497a-8de2-a3f4f6350a20 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:47:22.743235 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b4166-4fdd-497a-8de2-a3f4f6350a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:47:22.743517 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b4166-4fdd-497a-8de2-a3f4f6350a20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:47:22.744272 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2b4166-4fdd-497a-8de2-a3f4f6350a20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:47:22.745946 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2b4166-4fdd-497a-8de2-a3f4f6350a20 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:47:22.760599 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1110/2234] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:47:22 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:47:22.765596 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55f62e0-c95d-4af4-b7d2-c0a1bf59ddd6 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:47:22.776961 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55f62e0-c95d-4af4-b7d2-c0a1bf59ddd6 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:47:22.782784 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee90f0-fd76-4cab-a646-bd3ffea8be3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:47:22.783546 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee90f0-fd76-4cab-a646-bd3ffea8be3d admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:47:22.783839 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee90f0-fd76-4cab-a646-bd3ffea8be3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:47:22.784109 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee90f0-fd76-4cab-a646-bd3ffea8be3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:47:22.784847 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bee90f0-fd76-4cab-a646-bd3ffea8be3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:47:22.787128 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bee90f0-fd76-4cab-a646-bd3ffea8be3d admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:47:22.798289 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1125/2235] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:47:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:47:22.802369 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b4166-4fdd-497a-8de2-a3f4f6350a20 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:47:22.815619 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b4166-4fdd-497a-8de2-a3f4f6350a20 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:47:22.821086 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570ca036-d342-4bcb-8e9d-e53a571fcae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:47:22.822053 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ca036-d342-4bcb-8e9d-e53a571fcae2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:47:22.822343 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ca036-d342-4bcb-8e9d-e53a571fcae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:47:22.822629 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ca036-d342-4bcb-8e9d-e53a571fcae2 admin admin] PATH_INFO: `/v3/projects/ec74d88ce39042d3967c7ea64c5d3726/users/81624d22806e45fb80252b34ae921fa9/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:47:22.839649 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570ca036-d342-4bcb-8e9d-e53a571fcae2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec74d88ce39042d3967c7ea64c5d3726, user_id=81624d22806e45fb80252b34ae921fa9, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:47:22.842382 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570ca036-d342-4bcb-8e9d-e53a571fcae2 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:47:22.859354 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1111/2236] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:47:22 2022] PUT /identity/v3/projects/ec74d88ce39042d3967c7ea64c5d3726/users/81624d22806e45fb80252b34ae921fa9/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:47:22.869942 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7067a4-679d-4fd2-ad5f-7131966533e4 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:47:22.870681 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7067a4-679d-4fd2-ad5f-7131966533e4 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:47:22.870960 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7067a4-679d-4fd2-ad5f-7131966533e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:47:22.904110 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-6e7067a4-679d-4fd2-ad5f-7131966533e4 None None] MFA Rules not processed for user `81624d22806e45fb80252b34ae921fa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:47:22.905743 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-6e7067a4-679d-4fd2-ad5f-7131966533e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:47:23.000344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1126/2237] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:47:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:47:23.012702 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570ca036-d342-4bcb-8e9d-e53a571fcae2 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:47:23.014487 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-570ca036-d342-4bcb-8e9d-e53a571fcae2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:47:23.082028 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570ca036-d342-4bcb-8e9d-e53a571fcae2 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:47:23.086993 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de59ae25-1d80-4812-a040-1928a3265709 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:47:23.087650 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de59ae25-1d80-4812-a040-1928a3265709 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:47:23.087939 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de59ae25-1d80-4812-a040-1928a3265709 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:47:23.088224 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de59ae25-1d80-4812-a040-1928a3265709 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:47:23.098203 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de59ae25-1d80-4812-a040-1928a3265709 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:47:23.100174 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de59ae25-1d80-4812-a040-1928a3265709 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:47:23.113201 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1112/2238] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:47:23 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:47:32.996382 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=88,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:47:33.106169 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=44,keystone:INSERT=2 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:47:44.285171 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dea852-caec-417b-a720-f191993ef260 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:47:44.285773 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dea852-caec-417b-a720-f191993ef260 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:47:44.285878 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dea852-caec-417b-a720-f191993ef260 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:47:44.288582 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1127/2239] 10.222.0.36 () {54 vars in 921 bytes} [Thu May 19 16:47:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:47:44.295593 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ff0d1-faa4-4647-9644-7e3d023f5509 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:47:44.296143 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ff0d1-faa4-4647-9644-7e3d023f5509 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:47:44.296143 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ff0d1-faa4-4647-9644-7e3d023f5509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:47:44.356372 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-c75ff0d1-faa4-4647-9644-7e3d023f5509 None None] MFA Rules not processed for user `58bdeada00b74b7b8e186e6dc51e6dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:47:44.357831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-c75ff0d1-faa4-4647-9644-7e3d023f5509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:47:44.446006 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1113/2240] 10.222.0.36 () {58 vars in 1018 bytes} [Thu May 19 16:47:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:47:44.459472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dea852-caec-417b-a720-f191993ef260 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:47:44.461243 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-30dea852-caec-417b-a720-f191993ef260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:47:44.550919 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dea852-caec-417b-a720-f191993ef260 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:47:44.555322 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7acc1f-87ee-470e-92a0-6a2581c30df8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57a5d7151478434f86be085bcc487f0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:47:44.555977 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7acc1f-87ee-470e-92a0-6a2581c30df8 service glance] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:47:44.556301 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7acc1f-87ee-470e-92a0-6a2581c30df8 service glance] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:47:44.556583 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7acc1f-87ee-470e-92a0-6a2581c30df8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:47:44.566960 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7acc1f-87ee-470e-92a0-6a2581c30df8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:47:44.568419 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7acc1f-87ee-470e-92a0-6a2581c30df8 service glance] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:47:44.579094 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1128/2241] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:47:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:47:44.645422 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc4637-e752-4ef4-947e-b70cadbddc5d None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:47:44.645716 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc4637-e752-4ef4-947e-b70cadbddc5d None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:47:44.645997 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc4637-e752-4ef4-947e-b70cadbddc5d None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:47:44.647542 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1114/2242] 10.222.0.36 () {54 vars in 921 bytes} [Thu May 19 16:47:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 16:47:44.652506 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8dc97e-1409-4662-8382-d2066a9dae1e None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:47:44.652815 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8dc97e-1409-4662-8382-d2066a9dae1e None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:47:44.653103 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8dc97e-1409-4662-8382-d2066a9dae1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:47:44.671882 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-4f8dc97e-1409-4662-8382-d2066a9dae1e None None] MFA Rules not processed for user `33472643b65e4fe08fd5b3aa2c618133`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:47:44.673088 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-4f8dc97e-1409-4662-8382-d2066a9dae1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:47:44.730203 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1129/2243] 10.222.0.36 () {58 vars in 1018 bytes} [Thu May 19 16:47:44 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:47:44.742353 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc4637-e752-4ef4-947e-b70cadbddc5d None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:47:44.744277 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-27bc4637-e752-4ef4-947e-b70cadbddc5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:47:44.758422 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc4637-e752-4ef4-947e-b70cadbddc5d None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:47:44.762110 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad300da-d661-4279-847c-77b49b87af64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:47:44.762745 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad300da-d661-4279-847c-77b49b87af64 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:47:44.763026 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad300da-d661-4279-847c-77b49b87af64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:47:44.763301 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad300da-d661-4279-847c-77b49b87af64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:47:44.770828 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad300da-d661-4279-847c-77b49b87af64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:47:44.772248 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad300da-d661-4279-847c-77b49b87af64 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:47:44.779924 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1115/2244] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:47:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:47:44.866347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63b9bd-029b-4101-a35b-44e9275628ef None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:47:44.866641 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63b9bd-029b-4101-a35b-44e9275628ef None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:47:44.866924 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63b9bd-029b-4101-a35b-44e9275628ef None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:47:44.868344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1130/2245] 10.222.0.36 () {54 vars in 921 bytes} [Thu May 19 16:47:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:47:44.873599 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac9a3a-215f-4a8d-9c1e-919407b80a37 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:47:44.873895 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac9a3a-215f-4a8d-9c1e-919407b80a37 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:47:44.874175 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac9a3a-215f-4a8d-9c1e-919407b80a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:47:44.892788 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-b4ac9a3a-215f-4a8d-9c1e-919407b80a37 None None] MFA Rules not processed for user `58bdeada00b74b7b8e186e6dc51e6dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:47:44.894041 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-b4ac9a3a-215f-4a8d-9c1e-919407b80a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:47:44.905576 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1116/2246] 10.222.0.36 () {58 vars in 1018 bytes} [Thu May 19 16:47:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:47:44.916546 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc63b9bd-029b-4101-a35b-44e9275628ef None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:47:44.928056 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc63b9bd-029b-4101-a35b-44e9275628ef None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:47:44.931694 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e19bed-810d-411c-a5a9-17b1fab27e54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:47:44.932332 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e19bed-810d-411c-a5a9-17b1fab27e54 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:47:44.932617 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e19bed-810d-411c-a5a9-17b1fab27e54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:47:44.932901 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e19bed-810d-411c-a5a9-17b1fab27e54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:47:44.943511 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e19bed-810d-411c-a5a9-17b1fab27e54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:47:44.944949 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e19bed-810d-411c-a5a9-17b1fab27e54 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:47:44.955929 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1131/2247] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:47:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:47:47.486659 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2dc181-b608-4021-b069-4c3728d267f5 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:47:47.487155 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2dc181-b608-4021-b069-4c3728d267f5 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:47:47.487252 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2dc181-b608-4021-b069-4c3728d267f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:47:47.507542 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-df2dc181-b608-4021-b069-4c3728d267f5 None None] MFA Rules not processed for user `a30dc1dd2b00463bb3c7957864c56043`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:47:47.511808 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-df2dc181-b608-4021-b069-4c3728d267f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:47:47.568316 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1117/2248] 10.222.0.36 () {58 vars in 1009 bytes} [Thu May 19 16:47:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:47:47.580117 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e19bed-810d-411c-a5a9-17b1fab27e54 service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:47:47.581550 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-51e19bed-810d-411c-a5a9-17b1fab27e54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:47:47.659437 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e19bed-810d-411c-a5a9-17b1fab27e54 service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:47:47.665113 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edc7a45-7334-4305-9e74-9c75f7cb7efd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffe7cebc6f3b40d8a8e1866ed2da9822', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:47:47.665793 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc7a45-7334-4305-9e74-9c75f7cb7efd service swift] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:47:47.666078 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc7a45-7334-4305-9e74-9c75f7cb7efd service swift] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:47:47.666357 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc7a45-7334-4305-9e74-9c75f7cb7efd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:47:47.674955 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edc7a45-7334-4305-9e74-9c75f7cb7efd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:47:47.677244 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edc7a45-7334-4305-9e74-9c75f7cb7efd service swift] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:47:47.685448 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1132/2249] 10.222.0.36 () {60 vars in 1384 bytes} [Thu May 19 16:47:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 16:47:47.820738 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2dc181-b608-4021-b069-4c3728d267f5 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:47:47.904704 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:47:47.941703 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2dc181-b608-4021-b069-4c3728d267f5 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:47:47.949078 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf63490a-e064-4c6e-9c26-dfa870fe5bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:47:47.950460 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf63490a-e064-4c6e-9c26-dfa870fe5bfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:47:47.950894 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf63490a-e064-4c6e-9c26-dfa870fe5bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:47:47.951314 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf63490a-e064-4c6e-9c26-dfa870fe5bfb admin admin] PATH_INFO: `/v3/users/81624d22806e45fb80252b34ae921fa9` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:47:47.953269 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf63490a-e064-4c6e-9c26-dfa870fe5bfb admin admin] RBAC: Authorizing `identity:delete_user(user_id=81624d22806e45fb80252b34ae921fa9)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:47:47.956659 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf63490a-e064-4c6e-9c26-dfa870fe5bfb admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:47:48.046383 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-bf63490a-e064-4c6e-9c26-dfa870fe5bfb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81624d22806e45fb80252b34ae921fa9'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:47:48.047177 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-bf63490a-e064-4c6e-9c26-dfa870fe5bfb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81624d22806e45fb80252b34ae921fa9'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:47:48.057910 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-bf63490a-e064-4c6e-9c26-dfa870fe5bfb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81624d22806e45fb80252b34ae921fa9'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:47:48.077531 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-bf63490a-e064-4c6e-9c26-dfa870fe5bfb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81624d22806e45fb80252b34ae921fa9'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:47:48.078550 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-bf63490a-e064-4c6e-9c26-dfa870fe5bfb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81624d22806e45fb80252b34ae921fa9'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:47:48.099924 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a6796-5318-4fdf-8c76-bce61c5bc177 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:47:48.100224 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a6796-5318-4fdf-8c76-bce61c5bc177 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:47:48.100494 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a6796-5318-4fdf-8c76-bce61c5bc177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:47:48.102496 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1118/2250] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:47:47 2022] DELETE /identity/v3/users/81624d22806e45fb80252b34ae921fa9 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:47:48.119576 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-e26a6796-5318-4fdf-8c76-bce61c5bc177 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:47:48.120794 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e26a6796-5318-4fdf-8c76-bce61c5bc177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:47:48.127539 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:47:48.199001 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1133/2251] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:47:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:47:48.209773 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf63490a-e064-4c6e-9c26-dfa870fe5bfb admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:47:48.211174 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-bf63490a-e064-4c6e-9c26-dfa870fe5bfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:47:48.280526 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf63490a-e064-4c6e-9c26-dfa870fe5bfb admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:47:48.286797 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe74bb-1d99-4762-8535-dd3c8ea12fb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:47:48.287747 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe74bb-1d99-4762-8535-dd3c8ea12fb1 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:47:48.288032 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe74bb-1d99-4762-8535-dd3c8ea12fb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:47:48.288311 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe74bb-1d99-4762-8535-dd3c8ea12fb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:47:48.298892 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe74bb-1d99-4762-8535-dd3c8ea12fb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:47:48.300363 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe74bb-1d99-4762-8535-dd3c8ea12fb1 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:47:48.313260 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1119/2252] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:47:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:47:48.620349 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a6796-5318-4fdf-8c76-bce61c5bc177 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:47:48.621964 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e26a6796-5318-4fdf-8c76-bce61c5bc177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:47:48.635312 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a6796-5318-4fdf-8c76-bce61c5bc177 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:47:48.649444 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10be4a39-d8ce-47eb-ae84-447f2d416e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:47:48.650975 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be4a39-d8ce-47eb-ae84-447f2d416e94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:47:48.651334 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be4a39-d8ce-47eb-ae84-447f2d416e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:47:48.651506 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be4a39-d8ce-47eb-ae84-447f2d416e94 admin admin] PATH_INFO: `/v3/projects/ec74d88ce39042d3967c7ea64c5d3726` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:47:48.652805 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10be4a39-d8ce-47eb-ae84-447f2d416e94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec74d88ce39042d3967c7ea64c5d3726)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:47:48.654857 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10be4a39-d8ce-47eb-ae84-447f2d416e94 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:47:48.705431 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-10be4a39-d8ce-47eb-ae84-447f2d416e94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec74d88ce39042d3967c7ea64c5d3726', 'request_id': 'req-10be4a39-d8ce-47eb-ae84-447f2d416e94'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:47:48.711767 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-10be4a39-d8ce-47eb-ae84-447f2d416e94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec74d88ce39042d3967c7ea64c5d3726', 'request_id': 'req-10be4a39-d8ce-47eb-ae84-447f2d416e94'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:47:48.718272 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-10be4a39-d8ce-47eb-ae84-447f2d416e94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec74d88ce39042d3967c7ea64c5d3726', 'request_id': 'req-10be4a39-d8ce-47eb-ae84-447f2d416e94'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:47:48.731601 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-10be4a39-d8ce-47eb-ae84-447f2d416e94 admin admin] The token cache is being invalidate because project ec74d88ce39042d3967c7ea64c5d3726 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:47:48.732008 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-10be4a39-d8ce-47eb-ae84-447f2d416e94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:47:48.732640 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-10be4a39-d8ce-47eb-ae84-447f2d416e94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:47:48.734207 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1134/2253] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:47:48 2022] DELETE /identity/v3/projects/ec74d88ce39042d3967c7ea64c5d3726 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:47:48.739521 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27648fee-2326-4671-ad71-61f329c8feaf None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:47:48.740030 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27648fee-2326-4671-ad71-61f329c8feaf None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:47:48.740030 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27648fee-2326-4671-ad71-61f329c8feaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:47:48.759587 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-27648fee-2326-4671-ad71-61f329c8feaf None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:47:48.760808 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-27648fee-2326-4671-ad71-61f329c8feaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:47:48.824049 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1120/2254] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:47:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:47:48.831616 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddca1477-72c3-4554-bec1-d8c9de23f71c None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:47:48.831913 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddca1477-72c3-4554-bec1-d8c9de23f71c None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:47:48.832197 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddca1477-72c3-4554-bec1-d8c9de23f71c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:47:48.850921 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-ddca1477-72c3-4554-bec1-d8c9de23f71c None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:47:48.852130 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-ddca1477-72c3-4554-bec1-d8c9de23f71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:47:48.860267 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1135/2255] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:47:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:47:48.864663 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27648fee-2326-4671-ad71-61f329c8feaf None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:47:48.875015 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27648fee-2326-4671-ad71-61f329c8feaf None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:47:48.880743 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3380f86-0d77-4ee7-aed5-229e06eb3f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:47:48.881443 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3380f86-0d77-4ee7-aed5-229e06eb3f02 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:47:48.881734 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3380f86-0d77-4ee7-aed5-229e06eb3f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:47:48.882015 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3380f86-0d77-4ee7-aed5-229e06eb3f02 admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:47:48.882755 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3380f86-0d77-4ee7-aed5-229e06eb3f02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:47:48.883068 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3380f86-0d77-4ee7-aed5-229e06eb3f02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:47:48.884694 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3380f86-0d77-4ee7-aed5-229e06eb3f02 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:47:48.886499 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:47:48.900924 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1121/2256] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:47:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:47:48.904979 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddca1477-72c3-4554-bec1-d8c9de23f71c None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:47:48.914819 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddca1477-72c3-4554-bec1-d8c9de23f71c None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:47:48.919612 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80f79ca-b985-423e-8d2d-2f1fc746b943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:47:48.920288 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f79ca-b985-423e-8d2d-2f1fc746b943 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:47:48.920577 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f79ca-b985-423e-8d2d-2f1fc746b943 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:47:48.920848 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f79ca-b985-423e-8d2d-2f1fc746b943 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:47:48.921630 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80f79ca-b985-423e-8d2d-2f1fc746b943 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:47:48.923561 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80f79ca-b985-423e-8d2d-2f1fc746b943 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:47:48.952160 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1136/2257] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:47:48 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:47:48.956559 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3380f86-0d77-4ee7-aed5-229e06eb3f02 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:47:48.966627 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3380f86-0d77-4ee7-aed5-229e06eb3f02 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:47:49.026362 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecf710e-a28f-4a39-aed8-531e9b4d0a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:47:49.027290 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf710e-a28f-4a39-aed8-531e9b4d0a45 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:47:49.027759 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf710e-a28f-4a39-aed8-531e9b4d0a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:47:49.028041 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf710e-a28f-4a39-aed8-531e9b4d0a45 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:47:49.028760 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecf710e-a28f-4a39-aed8-531e9b4d0a45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:47:49.030751 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecf710e-a28f-4a39-aed8-531e9b4d0a45 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:47:49.055090 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1122/2258] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:47:48 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:47:49.056959 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80f79ca-b985-423e-8d2d-2f1fc746b943 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:47:49.067661 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80f79ca-b985-423e-8d2d-2f1fc746b943 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:47:49.072350 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d692a1-102b-4a92-9ba5-ce5fda35e2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:47:49.073030 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d692a1-102b-4a92-9ba5-ce5fda35e2ce admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:47:49.073352 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d692a1-102b-4a92-9ba5-ce5fda35e2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:47:49.073631 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d692a1-102b-4a92-9ba5-ce5fda35e2ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:47:49.074377 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d692a1-102b-4a92-9ba5-ce5fda35e2ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:47:49.076010 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d692a1-102b-4a92-9ba5-ce5fda35e2ce admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:47:49.087129 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1137/2259] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:47:49.091041 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecf710e-a28f-4a39-aed8-531e9b4d0a45 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:47:49.101724 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecf710e-a28f-4a39-aed8-531e9b4d0a45 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:47:49.106551 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44b2f3-daad-440f-b108-2f7258faa1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:47:49.107230 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44b2f3-daad-440f-b108-2f7258faa1bd admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:47:49.107517 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44b2f3-daad-440f-b108-2f7258faa1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:47:49.107798 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44b2f3-daad-440f-b108-2f7258faa1bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:47:49.108528 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a44b2f3-daad-440f-b108-2f7258faa1bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:47:49.110173 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a44b2f3-daad-440f-b108-2f7258faa1bd admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:47:49.121490 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1123/2260] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:47:49.125973 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d692a1-102b-4a92-9ba5-ce5fda35e2ce admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:47:49.136759 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d692a1-102b-4a92-9ba5-ce5fda35e2ce admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:47:49.142445 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec9e57e-d807-4b3f-8ea1-bd3725322566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:47:49.143334 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec9e57e-d807-4b3f-8ea1-bd3725322566 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:47:49.143621 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec9e57e-d807-4b3f-8ea1-bd3725322566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:47:49.143903 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec9e57e-d807-4b3f-8ea1-bd3725322566 admin admin] PATH_INFO: `/v3/projects/66c1f23fbe114e9885266a0092cca13c/users/39a11fe9bc8549639e7e13b6f490641c/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:47:49.159318 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec9e57e-d807-4b3f-8ea1-bd3725322566 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66c1f23fbe114e9885266a0092cca13c, user_id=39a11fe9bc8549639e7e13b6f490641c, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:47:49.162588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec9e57e-d807-4b3f-8ea1-bd3725322566 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:47:49.177283 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1138/2261] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:47:49 2022] PUT /identity/v3/projects/66c1f23fbe114e9885266a0092cca13c/users/39a11fe9bc8549639e7e13b6f490641c/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:47:49.184021 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a1daf-862f-4bc5-a570-ab34ad66be50 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:47:49.184410 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a1daf-862f-4bc5-a570-ab34ad66be50 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:47:49.185042 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a1daf-862f-4bc5-a570-ab34ad66be50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:47:49.206238 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-d19a1daf-862f-4bc5-a570-ab34ad66be50 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:47:49.208090 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-d19a1daf-862f-4bc5-a570-ab34ad66be50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:47:49.333730 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1124/2262] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:47:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:47:49.352140 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec9e57e-d807-4b3f-8ea1-bd3725322566 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:47:49.352140 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-eec9e57e-d807-4b3f-8ea1-bd3725322566 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:47:49.390448 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:47:49.421255 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec9e57e-d807-4b3f-8ea1-bd3725322566 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:47:49.425029 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d956d72-86a2-4f0d-925a-e4a48788e57a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:47:49.425728 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d956d72-86a2-4f0d-925a-e4a48788e57a service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:47:49.426015 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d956d72-86a2-4f0d-925a-e4a48788e57a service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:47:49.426332 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d956d72-86a2-4f0d-925a-e4a48788e57a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:47:49.444023 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d956d72-86a2-4f0d-925a-e4a48788e57a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:47:49.445649 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d956d72-86a2-4f0d-925a-e4a48788e57a service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:47:49.456695 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1139/2263] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:47:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:47:54.566377 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c1c06-9c5e-49e1-9cd0-b806ed36c278 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:47:54.567548 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c1c06-9c5e-49e1-9cd0-b806ed36c278 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:47:54.568017 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c1c06-9c5e-49e1-9cd0-b806ed36c278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:47:54.598749 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-9d3c1c06-9c5e-49e1-9cd0-b806ed36c278 None None] MFA Rules not processed for user `39a11fe9bc8549639e7e13b6f490641c`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:47:54.599981 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-9d3c1c06-9c5e-49e1-9cd0-b806ed36c278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:47:54.770774 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1125/2264] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:47:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:47:54.775994 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d956d72-86a2-4f0d-925a-e4a48788e57a service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:47:54.786890 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d956d72-86a2-4f0d-925a-e4a48788e57a service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:47:54.792935 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a2277c-4b17-479a-9425-afa5689febf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:47:54.801896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2277c-4b17-479a-9425-afa5689febf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:47:54.802587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2277c-4b17-479a-9425-afa5689febf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:47:54.802981 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2277c-4b17-479a-9425-afa5689febf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:47:54.803965 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a2277c-4b17-479a-9425-afa5689febf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:47:54.806421 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a2277c-4b17-479a-9425-afa5689febf7 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:47:54.855583 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1140/2265] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:47:54 2022] POST /identity/v3/projects => generated 385 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:47:54.856895 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3c1c06-9c5e-49e1-9cd0-b806ed36c278 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:47:54.863093 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:47:54.873644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3c1c06-9c5e-49e1-9cd0-b806ed36c278 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:47:54.954393 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2caebad-85bd-4620-91f8-ceb1e5eb742b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:47:54.955006 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2caebad-85bd-4620-91f8-ceb1e5eb742b admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:47:54.957495 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2caebad-85bd-4620-91f8-ceb1e5eb742b admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:47:54.957916 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2caebad-85bd-4620-91f8-ceb1e5eb742b admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:47:54.958766 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2caebad-85bd-4620-91f8-ceb1e5eb742b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:47:54.960781 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2caebad-85bd-4620-91f8-ceb1e5eb742b admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:47:54.987901 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1126/2266] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:47:54 2022] POST /identity/v3/users => generated 299 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:47:54.993192 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a2277c-4b17-479a-9425-afa5689febf7 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:47:55.005757 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a2277c-4b17-479a-9425-afa5689febf7 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:47:55.012910 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8150090f-8327-4f37-9bcf-862b20e9b54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:47:55.013678 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8150090f-8327-4f37-9bcf-862b20e9b54e admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:47:55.013973 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8150090f-8327-4f37-9bcf-862b20e9b54e admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:47:55.014256 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8150090f-8327-4f37-9bcf-862b20e9b54e admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:47:55.015479 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8150090f-8327-4f37-9bcf-862b20e9b54e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:47:55.016658 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8150090f-8327-4f37-9bcf-862b20e9b54e admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:47:55.035170 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1141/2267] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:47:54 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:47:55.038289 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2caebad-85bd-4620-91f8-ceb1e5eb742b admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:47:55.054352 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2caebad-85bd-4620-91f8-ceb1e5eb742b admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:47:55.067731 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e1f859-fcf9-4ae2-a089-f067f4b044db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:47:55.068865 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e1f859-fcf9-4ae2-a089-f067f4b044db admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:47:55.069173 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e1f859-fcf9-4ae2-a089-f067f4b044db admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:47:55.073617 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e1f859-fcf9-4ae2-a089-f067f4b044db admin admin] PATH_INFO: `/v3/projects/6c47797a5c954e76a8a79974b42df0d0/users/2af2f8e4af214c64a3681b7b6b3fd13c/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:47:55.091963 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e1f859-fcf9-4ae2-a089-f067f4b044db admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c47797a5c954e76a8a79974b42df0d0, user_id=2af2f8e4af214c64a3681b7b6b3fd13c, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:47:55.094961 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e1f859-fcf9-4ae2-a089-f067f4b044db admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:47:55.109037 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1127/2268] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:47:55 2022] PUT /identity/v3/projects/6c47797a5c954e76a8a79974b42df0d0/users/2af2f8e4af214c64a3681b7b6b3fd13c/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:47:55.112925 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8150090f-8327-4f37-9bcf-862b20e9b54e admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:47:55.123630 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8150090f-8327-4f37-9bcf-862b20e9b54e admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:47:55.184324 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be594eda-41cc-4ae5-9844-caa427219add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:47:55.185007 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be594eda-41cc-4ae5-9844-caa427219add admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:47:55.185412 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be594eda-41cc-4ae5-9844-caa427219add admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:47:55.185794 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be594eda-41cc-4ae5-9844-caa427219add admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:47:55.186626 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be594eda-41cc-4ae5-9844-caa427219add admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:47:55.188619 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be594eda-41cc-4ae5-9844-caa427219add admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:47:55.202816 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1142/2269] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:47:55 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:47:55.206476 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e1f859-fcf9-4ae2-a089-f067f4b044db admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:47:55.217880 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e1f859-fcf9-4ae2-a089-f067f4b044db admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:47:55.223520 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9352b782-8c69-4af5-81b0-2ff56f2ac14b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:47:55.224530 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352b782-8c69-4af5-81b0-2ff56f2ac14b admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:47:55.224826 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352b782-8c69-4af5-81b0-2ff56f2ac14b admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:47:55.225092 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352b782-8c69-4af5-81b0-2ff56f2ac14b admin admin] PATH_INFO: `/v3/projects/6c47797a5c954e76a8a79974b42df0d0/users/2af2f8e4af214c64a3681b7b6b3fd13c/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:47:55.227132 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9352b782-8c69-4af5-81b0-2ff56f2ac14b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c47797a5c954e76a8a79974b42df0d0, user_id=2af2f8e4af214c64a3681b7b6b3fd13c, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:47:55.229925 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9352b782-8c69-4af5-81b0-2ff56f2ac14b admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:47:55.241466 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1128/2270] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:47:55 2022] PUT /identity/v3/projects/6c47797a5c954e76a8a79974b42df0d0/users/2af2f8e4af214c64a3681b7b6b3fd13c/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:48:00.195310 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be594eda-41cc-4ae5-9844-caa427219add admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:48:00.197567 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-be594eda-41cc-4ae5-9844-caa427219add admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:48:00.289036 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be594eda-41cc-4ae5-9844-caa427219add admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:48:00.353215 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e7bbf4-edae-42b4-a974-cfb4336b4508 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:48:00.353971 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e7bbf4-edae-42b4-a974-cfb4336b4508 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:48:00.354352 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e7bbf4-edae-42b4-a974-cfb4336b4508 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:48:00.354734 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e7bbf4-edae-42b4-a974-cfb4336b4508 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:48:00.356640 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-16e7bbf4-edae-42b4-a974-cfb4336b4508 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:48:00.361841 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:48:00.462045 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e7bbf4-edae-42b4-a974-cfb4336b4508 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:48:00.464051 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e7bbf4-edae-42b4-a974-cfb4336b4508 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:48:00.479034 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1143/2271] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:48:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:48:01.157264 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d37ac-d3fb-40aa-baf1-3e061959d77e None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:48:01.158332 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d37ac-d3fb-40aa-baf1-3e061959d77e None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:48:01.158623 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d37ac-d3fb-40aa-baf1-3e061959d77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:48:01.192126 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-707d37ac-d3fb-40aa-baf1-3e061959d77e None None] MFA Rules not processed for user `2af2f8e4af214c64a3681b7b6b3fd13c`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:48:01.193729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-707d37ac-d3fb-40aa-baf1-3e061959d77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:48:01.285541 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1129/2272] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:48:01 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:48:01.297613 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e7bbf4-edae-42b4-a974-cfb4336b4508 service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:48:01.299087 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-16e7bbf4-edae-42b4-a974-cfb4336b4508 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:48:01.364218 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e7bbf4-edae-42b4-a974-cfb4336b4508 service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:48:01.367941 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ca3072-92f8-4faa-8c8c-8439d998a21c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57a5d7151478434f86be085bcc487f0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:48:01.368604 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ca3072-92f8-4faa-8c8c-8439d998a21c service glance] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:48:01.368886 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ca3072-92f8-4faa-8c8c-8439d998a21c service glance] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:48:01.369201 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ca3072-92f8-4faa-8c8c-8439d998a21c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:48:01.430245 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ca3072-92f8-4faa-8c8c-8439d998a21c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:48:01.432034 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ca3072-92f8-4faa-8c8c-8439d998a21c service glance] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:48:01.443757 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1144/2273] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:48:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:48:01.547391 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8403c4f-693e-4cba-af9b-960ee1fe8cf6 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:48:01.547765 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8403c4f-693e-4cba-af9b-960ee1fe8cf6 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:48:01.548250 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8403c4f-693e-4cba-af9b-960ee1fe8cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:48:01.571126 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-e8403c4f-693e-4cba-af9b-960ee1fe8cf6 None None] MFA Rules not processed for user `a30dc1dd2b00463bb3c7957864c56043`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:48:01.572544 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-e8403c4f-693e-4cba-af9b-960ee1fe8cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:48:01.610257 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:48:01.635237 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1130/2274] 10.222.0.36 () {58 vars in 1009 bytes} [Thu May 19 16:48:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:48:01.647458 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ca3072-92f8-4faa-8c8c-8439d998a21c service glance] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:48:01.648889 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b8ca3072-92f8-4faa-8c8c-8439d998a21c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:48:01.704090 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:48:01.726759 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ca3072-92f8-4faa-8c8c-8439d998a21c service glance] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:48:01.732001 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f4fd01-3e24-4157-8ec8-ec5b2a7b3c92 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffe7cebc6f3b40d8a8e1866ed2da9822', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:48:01.732680 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4fd01-3e24-4157-8ec8-ec5b2a7b3c92 service swift] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:48:01.732968 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4fd01-3e24-4157-8ec8-ec5b2a7b3c92 service swift] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:48:01.733295 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4fd01-3e24-4157-8ec8-ec5b2a7b3c92 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:48:01.743018 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f4fd01-3e24-4157-8ec8-ec5b2a7b3c92 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:48:01.744655 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f4fd01-3e24-4157-8ec8-ec5b2a7b3c92 service swift] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:48:01.756289 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1145/2275] 10.222.0.36 () {60 vars in 1384 bytes} [Thu May 19 16:48:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 16:48:01.989735 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8403c4f-693e-4cba-af9b-960ee1fe8cf6 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:48:02.002634 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8403c4f-693e-4cba-af9b-960ee1fe8cf6 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:48:02.009208 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94169ba-859b-4f70-b31c-222e3505f1d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:48:02.011880 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94169ba-859b-4f70-b31c-222e3505f1d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:48:02.012172 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94169ba-859b-4f70-b31c-222e3505f1d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:48:02.012454 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94169ba-859b-4f70-b31c-222e3505f1d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:48:02.014417 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-d94169ba-859b-4f70-b31c-222e3505f1d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:48:02.031328 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94169ba-859b-4f70-b31c-222e3505f1d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:48:02.032797 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94169ba-859b-4f70-b31c-222e3505f1d1 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:48:02.045507 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1131/2276] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:48:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:48:11.061056 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da6134-460c-418b-b3c2-5e2a38203050 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:48:11.061557 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da6134-460c-418b-b3c2-5e2a38203050 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:48:11.061646 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da6134-460c-418b-b3c2-5e2a38203050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:48:11.078956 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-c9da6134-460c-418b-b3c2-5e2a38203050 None None] MFA Rules not processed for user `39a11fe9bc8549639e7e13b6f490641c`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:48:11.080178 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-c9da6134-460c-418b-b3c2-5e2a38203050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:48:11.088216 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1146/2277] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:48:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:48:11.342696 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94169ba-859b-4f70-b31c-222e3505f1d1 service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:48:11.351897 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94169ba-859b-4f70-b31c-222e3505f1d1 service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:48:11.355525 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64f1986-7265-4872-bc97-de96846cadae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:48:11.356168 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f1986-7265-4872-bc97-de96846cadae service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:48:11.356453 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f1986-7265-4872-bc97-de96846cadae service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:48:11.356733 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f1986-7265-4872-bc97-de96846cadae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:48:11.358544 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-d64f1986-7265-4872-bc97-de96846cadae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:48:11.368638 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64f1986-7265-4872-bc97-de96846cadae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:48:11.370097 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64f1986-7265-4872-bc97-de96846cadae service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:48:11.378393 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1132/2278] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:48:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:48:11.687832 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da6134-460c-418b-b3c2-5e2a38203050 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:48:11.689249 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-c9da6134-460c-418b-b3c2-5e2a38203050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:48:11.765887 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da6134-460c-418b-b3c2-5e2a38203050 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:48:11.771482 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2490250a-b170-4276-8082-c580506b21c4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e158e5c87f4954aeb297f42dabdc16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:48:11.772129 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490250a-b170-4276-8082-c580506b21c4 service placement] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:48:11.772414 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490250a-b170-4276-8082-c580506b21c4 service placement] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:48:11.772691 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490250a-b170-4276-8082-c580506b21c4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:48:11.774495 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-2490250a-b170-4276-8082-c580506b21c4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:48:11.786493 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2490250a-b170-4276-8082-c580506b21c4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:48:11.788079 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2490250a-b170-4276-8082-c580506b21c4 service placement] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:48:11.797723 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1147/2279] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:48:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:48:13.032035 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc4a51-8897-4653-a3be-033d09ac5253 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:48:13.032525 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc4a51-8897-4653-a3be-033d09ac5253 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:48:13.032628 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc4a51-8897-4653-a3be-033d09ac5253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:48:13.051648 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-9fbc4a51-8897-4653-a3be-033d09ac5253 None None] MFA Rules not processed for user `a30dc1dd2b00463bb3c7957864c56043`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:48:13.052905 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-9fbc4a51-8897-4653-a3be-033d09ac5253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:48:13.060607 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1133/2280] 10.222.0.36 () {58 vars in 1009 bytes} [Thu May 19 16:48:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:48:13.073892 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2490250a-b170-4276-8082-c580506b21c4 service placement] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:48:13.084939 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2490250a-b170-4276-8082-c580506b21c4 service placement] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:48:13.089984 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd79aae-602f-4259-9739-6e710465f5c5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffe7cebc6f3b40d8a8e1866ed2da9822', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:48:13.090662 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd79aae-602f-4259-9739-6e710465f5c5 service swift] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:48:13.090950 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd79aae-602f-4259-9739-6e710465f5c5 service swift] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:48:13.091234 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd79aae-602f-4259-9739-6e710465f5c5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:48:13.100224 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd79aae-602f-4259-9739-6e710465f5c5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:48:13.101864 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd79aae-602f-4259-9739-6e710465f5c5 service swift] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:48:13.112027 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1148/2281] 10.222.0.36 () {60 vars in 1384 bytes} [Thu May 19 16:48:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 16:48:16.971261 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc4a51-8897-4653-a3be-033d09ac5253 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:48:16.972810 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-9fbc4a51-8897-4653-a3be-033d09ac5253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:48:16.989348 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc4a51-8897-4653-a3be-033d09ac5253 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:48:16.994924 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb82b6d-3732-4afc-b42b-2bbb3f84cbc2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:48:16.995582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb82b6d-3732-4afc-b42b-2bbb3f84cbc2 service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:48:16.995867 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb82b6d-3732-4afc-b42b-2bbb3f84cbc2 service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:48:16.996144 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb82b6d-3732-4afc-b42b-2bbb3f84cbc2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:48:16.998917 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-bcb82b6d-3732-4afc-b42b-2bbb3f84cbc2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:48:17.066575 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb82b6d-3732-4afc-b42b-2bbb3f84cbc2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:48:17.068552 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb82b6d-3732-4afc-b42b-2bbb3f84cbc2 service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:48:17.078750 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1134/2282] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:48:16 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:48:23.108717 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=52 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:48:24.890036 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd79aae-602f-4259-9739-6e710465f5c5 service swift] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:48:24.901911 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd79aae-602f-4259-9739-6e710465f5c5 service swift] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:48:24.907221 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3cf349-09cf-441f-80c0-d0def1a49dcf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:48:24.907861 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3cf349-09cf-441f-80c0-d0def1a49dcf service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:48:24.908149 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3cf349-09cf-441f-80c0-d0def1a49dcf service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:48:24.908425 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3cf349-09cf-441f-80c0-d0def1a49dcf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:48:24.910245 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-9b3cf349-09cf-441f-80c0-d0def1a49dcf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:48:24.969923 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3cf349-09cf-441f-80c0-d0def1a49dcf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:48:24.971680 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3cf349-09cf-441f-80c0-d0def1a49dcf service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:48:24.981470 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1149/2283] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:48:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:48:27.073851 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=50 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:48:27.819386 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb82b6d-3732-4afc-b42b-2bbb3f84cbc2 service nova] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:48:27.828924 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb82b6d-3732-4afc-b42b-2bbb3f84cbc2 service nova] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:48:27.833332 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e4bb3a-1a24-463d-a1be-4ddc7a470006 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:48:27.834040 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4bb3a-1a24-463d-a1be-4ddc7a470006 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:48:27.834735 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4bb3a-1a24-463d-a1be-4ddc7a470006 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:48:27.834735 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4bb3a-1a24-463d-a1be-4ddc7a470006 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:48:27.836499 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-94e4bb3a-1a24-463d-a1be-4ddc7a470006 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:48:27.847667 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e4bb3a-1a24-463d-a1be-4ddc7a470006 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:48:27.849237 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e4bb3a-1a24-463d-a1be-4ddc7a470006 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:48:27.859666 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1135/2284] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:48:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:48:31.170453 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72200e41-c6c7-4646-acf9-2d9b71b0d7f0 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:48:31.170973 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72200e41-c6c7-4646-acf9-2d9b71b0d7f0 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:48:31.171179 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72200e41-c6c7-4646-acf9-2d9b71b0d7f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:48:31.189897 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-72200e41-c6c7-4646-acf9-2d9b71b0d7f0 None None] MFA Rules not processed for user `a30dc1dd2b00463bb3c7957864c56043`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:48:31.191118 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-72200e41-c6c7-4646-acf9-2d9b71b0d7f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:48:31.198939 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1150/2285] 10.222.0.36 () {58 vars in 967 bytes} [Thu May 19 16:48:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:48:31.221348 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e4bb3a-1a24-463d-a1be-4ddc7a470006 service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:48:31.234337 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e4bb3a-1a24-463d-a1be-4ddc7a470006 service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:48:31.240281 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e88773-69f2-4084-8d7d-83da4fc47dc9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffe7cebc6f3b40d8a8e1866ed2da9822', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:48:31.240937 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e88773-69f2-4084-8d7d-83da4fc47dc9 service swift] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:48:31.241259 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e88773-69f2-4084-8d7d-83da4fc47dc9 service swift] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:48:31.241549 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e88773-69f2-4084-8d7d-83da4fc47dc9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:48:31.250715 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e88773-69f2-4084-8d7d-83da4fc47dc9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:48:31.252306 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e88773-69f2-4084-8d7d-83da4fc47dc9 service swift] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:48:31.262486 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1136/2286] 10.222.0.36 () {60 vars in 1384 bytes} [Thu May 19 16:48:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 16:48:41.185064 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=31 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:48:41.258832 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=12 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:48:51.083194 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72200e41-c6c7-4646-acf9-2d9b71b0d7f0 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:48:51.092433 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72200e41-c6c7-4646-acf9-2d9b71b0d7f0 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:48:51.096186 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27867a69-3736-40e6-95a8-a5fd0587bb76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:48:51.096829 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27867a69-3736-40e6-95a8-a5fd0587bb76 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:48:51.097112 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27867a69-3736-40e6-95a8-a5fd0587bb76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:48:51.097421 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27867a69-3736-40e6-95a8-a5fd0587bb76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:48:51.106324 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27867a69-3736-40e6-95a8-a5fd0587bb76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:48:51.107771 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27867a69-3736-40e6-95a8-a5fd0587bb76 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:48:51.118144 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1151/2287] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:48:51 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:48:52.938641 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e88773-69f2-4084-8d7d-83da4fc47dc9 service swift] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:48:52.950907 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e88773-69f2-4084-8d7d-83da4fc47dc9 service swift] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:48:52.956992 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a69540a-da19-4cb2-a754-01f93a21e8dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:48:52.957904 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69540a-da19-4cb2-a754-01f93a21e8dc service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:48:52.958194 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69540a-da19-4cb2-a754-01f93a21e8dc service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:48:52.958476 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69540a-da19-4cb2-a754-01f93a21e8dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:48:52.960325 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-4a69540a-da19-4cb2-a754-01f93a21e8dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:48:52.972925 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a69540a-da19-4cb2-a754-01f93a21e8dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:48:52.974292 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a69540a-da19-4cb2-a754-01f93a21e8dc service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:48:52.987838 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1137/2288] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:48:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:49:01.113108 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=6 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:49:02.982250 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=6 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:49:06.336339 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27867a69-3736-40e6-95a8-a5fd0587bb76 service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:49:06.348498 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27867a69-3736-40e6-95a8-a5fd0587bb76 service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:49:06.354661 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2949bc-90af-4b64-8769-ac07a4212258 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:49:06.355302 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2949bc-90af-4b64-8769-ac07a4212258 service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:49:06.355595 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2949bc-90af-4b64-8769-ac07a4212258 service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:49:06.355882 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2949bc-90af-4b64-8769-ac07a4212258 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:49:06.357720 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-7f2949bc-90af-4b64-8769-ac07a4212258 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:49:06.416321 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2949bc-90af-4b64-8769-ac07a4212258 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:49:06.417961 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2949bc-90af-4b64-8769-ac07a4212258 service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:49:06.426977 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1152/2289] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:49:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:49:08.652027 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdee609-0e44-4cca-b570-fa2390fac3c7 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:49:08.652619 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdee609-0e44-4cca-b570-fa2390fac3c7 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:49:08.652619 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdee609-0e44-4cca-b570-fa2390fac3c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:49:08.683396 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-ebdee609-0e44-4cca-b570-fa2390fac3c7 None None] MFA Rules not processed for user `6e88db1bf9294decaf171756772a5066`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:49:08.684847 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ebdee609-0e44-4cca-b570-fa2390fac3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:49:08.694662 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1138/2290] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:49:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:49:16.422792 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=28 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:49:18.666650 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:49:21.478130 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dffb99-ab0e-420c-9aed-1678af2319b8 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:49:21.478645 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dffb99-ab0e-420c-9aed-1678af2319b8 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:49:21.478746 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dffb99-ab0e-420c-9aed-1678af2319b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:49:21.496552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-31dffb99-ab0e-420c-9aed-1678af2319b8 None None] MFA Rules not processed for user `0695c868a2d844089fcda29ea3653d5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:49:21.497792 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-31dffb99-ab0e-420c-9aed-1678af2319b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:49:21.504873 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1153/2291] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:49:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:49:21.810230 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdee609-0e44-4cca-b570-fa2390fac3c7 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:49:21.819738 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdee609-0e44-4cca-b570-fa2390fac3c7 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:49:21.823888 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b3065c-4045-46ee-9f4f-e66719052605 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:49:21.824524 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3065c-4045-46ee-9f4f-e66719052605 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:49:21.824810 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3065c-4045-46ee-9f4f-e66719052605 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:49:21.825087 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3065c-4045-46ee-9f4f-e66719052605 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:49:21.826888 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-d4b3065c-4045-46ee-9f4f-e66719052605 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:49:21.836170 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b3065c-4045-46ee-9f4f-e66719052605 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:49:21.838122 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b3065c-4045-46ee-9f4f-e66719052605 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:49:21.845889 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1139/2292] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:49:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 16:49:31.490629 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:49:31.781297 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dffb99-ab0e-420c-9aed-1678af2319b8 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:49:31.782768 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-31dffb99-ab0e-420c-9aed-1678af2319b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:49:31.800458 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dffb99-ab0e-420c-9aed-1678af2319b8 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:49:31.807256 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a174dfab-aeed-4448-b172-d1c68157282c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:49:31.808657 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a174dfab-aeed-4448-b172-d1c68157282c service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:49:31.808944 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a174dfab-aeed-4448-b172-d1c68157282c service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:49:31.809263 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a174dfab-aeed-4448-b172-d1c68157282c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:49:31.811136 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-a174dfab-aeed-4448-b172-d1c68157282c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:49:31.823911 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a174dfab-aeed-4448-b172-d1c68157282c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:49:31.825571 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a174dfab-aeed-4448-b172-d1c68157282c service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:49:31.836505 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1154/2293] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:49:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:49:31.843000 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=6 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:49:41.832499 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=6 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:49:49.253751 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c97d5-36ca-4a2c-9bb1-4dd724091a3d None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:49:49.254225 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c97d5-36ca-4a2c-9bb1-4dd724091a3d None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:49:49.254375 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c97d5-36ca-4a2c-9bb1-4dd724091a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:49:49.271532 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-551c97d5-36ca-4a2c-9bb1-4dd724091a3d None None] MFA Rules not processed for user `86b78ffb46894509bf8e81a154073885`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:49:49.272753 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-551c97d5-36ca-4a2c-9bb1-4dd724091a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:49:49.279684 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1140/2294] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:49:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:49:57.541383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a174dfab-aeed-4448-b172-d1c68157282c service nova] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:49:57.552518 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a174dfab-aeed-4448-b172-d1c68157282c service nova] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:49:57.557977 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d58386-904e-42d3-9bcf-ddab0dc45892 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:49:57.558612 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d58386-904e-42d3-9bcf-ddab0dc45892 service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:49:57.558897 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d58386-904e-42d3-9bcf-ddab0dc45892 service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:49:57.559176 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d58386-904e-42d3-9bcf-ddab0dc45892 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:49:57.560920 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b8d58386-904e-42d3-9bcf-ddab0dc45892 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:49:57.619921 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d58386-904e-42d3-9bcf-ddab0dc45892 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:49:57.621555 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d58386-904e-42d3-9bcf-ddab0dc45892 service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:49:57.630617 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1155/2295] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:49:57 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:49:59.266365 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:50:07.626548 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=28 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:50:41.017350 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c97d5-36ca-4a2c-9bb1-4dd724091a3d None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:41.027355 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c97d5-36ca-4a2c-9bb1-4dd724091a3d None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:41.031053 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d684cfd0-e02b-450f-9e6b-57bf4c70e174 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:41.031715 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684cfd0-e02b-450f-9e6b-57bf4c70e174 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:41.032007 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684cfd0-e02b-450f-9e6b-57bf4c70e174 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:41.032292 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684cfd0-e02b-450f-9e6b-57bf4c70e174 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:41.034294 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-d684cfd0-e02b-450f-9e6b-57bf4c70e174 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:50:41.097730 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d684cfd0-e02b-450f-9e6b-57bf4c70e174 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:41.099179 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d684cfd0-e02b-450f-9e6b-57bf4c70e174 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:41.108870 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1141/2296] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:50:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:50:44.432256 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d58386-904e-42d3-9bcf-ddab0dc45892 service nova] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:44.433821 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b8d58386-904e-42d3-9bcf-ddab0dc45892 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:50:44.450526 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d58386-904e-42d3-9bcf-ddab0dc45892 service nova] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:44.459211 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41fcc12-5306-4958-8947-086ff0eb1152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:44.461005 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41fcc12-5306-4958-8947-086ff0eb1152 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:44.461509 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41fcc12-5306-4958-8947-086ff0eb1152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:44.462523 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41fcc12-5306-4958-8947-086ff0eb1152 admin admin] PATH_INFO: `/v3/users/6e88db1bf9294decaf171756772a5066` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:44.464141 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41fcc12-5306-4958-8947-086ff0eb1152 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e88db1bf9294decaf171756772a5066)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:44.467158 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41fcc12-5306-4958-8947-086ff0eb1152 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:44.515188 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a41fcc12-5306-4958-8947-086ff0eb1152 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e88db1bf9294decaf171756772a5066'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:44.515825 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a41fcc12-5306-4958-8947-086ff0eb1152 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e88db1bf9294decaf171756772a5066'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:44.516434 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a41fcc12-5306-4958-8947-086ff0eb1152 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e88db1bf9294decaf171756772a5066'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:44.523544 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a41fcc12-5306-4958-8947-086ff0eb1152 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e88db1bf9294decaf171756772a5066'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:44.535268 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a41fcc12-5306-4958-8947-086ff0eb1152 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e88db1bf9294decaf171756772a5066'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:44.548630 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1156/2297] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:50:44 2022] DELETE /identity/v3/users/6e88db1bf9294decaf171756772a5066 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:50:44.731204 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d684cfd0-e02b-450f-9e6b-57bf4c70e174 service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:44.755997 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d684cfd0-e02b-450f-9e6b-57bf4c70e174 service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:44.819259 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929bad98-28fd-4ab9-97ad-3e74102f9aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:44.820130 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929bad98-28fd-4ab9-97ad-3e74102f9aef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:44.820456 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929bad98-28fd-4ab9-97ad-3e74102f9aef admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:44.820698 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929bad98-28fd-4ab9-97ad-3e74102f9aef admin admin] PATH_INFO: `/v3/projects/3fbe05607bf945bbb756e7ed3764cccc` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:44.821944 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929bad98-28fd-4ab9-97ad-3e74102f9aef admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fbe05607bf945bbb756e7ed3764cccc)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:44.823853 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929bad98-28fd-4ab9-97ad-3e74102f9aef admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:44.874323 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-929bad98-28fd-4ab9-97ad-3e74102f9aef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fbe05607bf945bbb756e7ed3764cccc', 'request_id': 'req-929bad98-28fd-4ab9-97ad-3e74102f9aef'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:44.880007 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-929bad98-28fd-4ab9-97ad-3e74102f9aef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fbe05607bf945bbb756e7ed3764cccc', 'request_id': 'req-929bad98-28fd-4ab9-97ad-3e74102f9aef'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:44.886173 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-929bad98-28fd-4ab9-97ad-3e74102f9aef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fbe05607bf945bbb756e7ed3764cccc', 'request_id': 'req-929bad98-28fd-4ab9-97ad-3e74102f9aef'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:44.900306 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-929bad98-28fd-4ab9-97ad-3e74102f9aef admin admin] The token cache is being invalidate because project 3fbe05607bf945bbb756e7ed3764cccc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:50:44.900594 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-929bad98-28fd-4ab9-97ad-3e74102f9aef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:44.901257 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-929bad98-28fd-4ab9-97ad-3e74102f9aef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:44.902824 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1142/2298] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:50:44 2022] DELETE /identity/v3/projects/3fbe05607bf945bbb756e7ed3764cccc => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:50:44.908278 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25afe1e-7c3f-4d0e-bd2a-1ac74d0001df None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:44.908596 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25afe1e-7c3f-4d0e-bd2a-1ac74d0001df None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:44.908911 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25afe1e-7c3f-4d0e-bd2a-1ac74d0001df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:44.927621 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-d25afe1e-7c3f-4d0e-bd2a-1ac74d0001df None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:50:44.928831 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d25afe1e-7c3f-4d0e-bd2a-1ac74d0001df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:50:44.988339 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1157/2299] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:50:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:50:44.996423 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98899d5-e376-4609-8461-9b98497ebda6 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:44.996982 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98899d5-e376-4609-8461-9b98497ebda6 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:44.997323 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98899d5-e376-4609-8461-9b98497ebda6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:45.015211 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-b98899d5-e376-4609-8461-9b98497ebda6 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:50:45.016411 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-b98899d5-e376-4609-8461-9b98497ebda6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:50:45.024539 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1143/2300] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:50:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:50:45.028613 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25afe1e-7c3f-4d0e-bd2a-1ac74d0001df None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:45.038950 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25afe1e-7c3f-4d0e-bd2a-1ac74d0001df None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:45.043593 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34869195-c864-4f77-ba40-5534e9bfdcd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:45.044252 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34869195-c864-4f77-ba40-5534e9bfdcd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:45.045252 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34869195-c864-4f77-ba40-5534e9bfdcd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:45.045568 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34869195-c864-4f77-ba40-5534e9bfdcd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:45.046307 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34869195-c864-4f77-ba40-5534e9bfdcd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:50:45.046627 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34869195-c864-4f77-ba40-5534e9bfdcd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:45.048282 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34869195-c864-4f77-ba40-5534e9bfdcd7 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:45.059831 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1158/2301] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:50:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:50:45.064814 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98899d5-e376-4609-8461-9b98497ebda6 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:45.074449 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98899d5-e376-4609-8461-9b98497ebda6 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:45.079732 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e72587-67be-4662-9453-a1a78e6cc2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:45.080419 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e72587-67be-4662-9453-a1a78e6cc2c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:45.080707 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e72587-67be-4662-9453-a1a78e6cc2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:45.080990 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e72587-67be-4662-9453-a1a78e6cc2c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:45.081757 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e72587-67be-4662-9453-a1a78e6cc2c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:45.083674 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e72587-67be-4662-9453-a1a78e6cc2c3 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:45.097940 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:50:45.120662 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1144/2302] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:50:45 2022] POST /identity/v3/projects => generated 365 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:50:45.125384 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34869195-c864-4f77-ba40-5534e9bfdcd7 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:45.135352 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34869195-c864-4f77-ba40-5534e9bfdcd7 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:45.190277 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ce9d9-c83f-4d51-8f99-b6d088dd30b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:45.190946 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ce9d9-c83f-4d51-8f99-b6d088dd30b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:45.191232 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ce9d9-c83f-4d51-8f99-b6d088dd30b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:45.191519 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ce9d9-c83f-4d51-8f99-b6d088dd30b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:45.192220 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5ce9d9-c83f-4d51-8f99-b6d088dd30b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:45.194209 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5ce9d9-c83f-4d51-8f99-b6d088dd30b8 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:45.201710 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:50:45.218910 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1159/2303] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:50:45 2022] POST /identity/v3/users => generated 283 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:50:45.222912 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e72587-67be-4662-9453-a1a78e6cc2c3 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:45.233272 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e72587-67be-4662-9453-a1a78e6cc2c3 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:45.237938 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4565ef34-13a7-4b46-970f-d546154e271d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:45.238652 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565ef34-13a7-4b46-970f-d546154e271d admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:45.238886 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565ef34-13a7-4b46-970f-d546154e271d admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:45.239168 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565ef34-13a7-4b46-970f-d546154e271d admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:45.239934 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4565ef34-13a7-4b46-970f-d546154e271d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:45.241586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4565ef34-13a7-4b46-970f-d546154e271d admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:45.251683 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1145/2304] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:50:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:50:45.255754 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ce9d9-c83f-4d51-8f99-b6d088dd30b8 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:45.266103 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ce9d9-c83f-4d51-8f99-b6d088dd30b8 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:45.270812 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a789a855-0ce3-4f25-9678-3b7a68018cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:45.271528 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789a855-0ce3-4f25-9678-3b7a68018cd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:45.271814 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789a855-0ce3-4f25-9678-3b7a68018cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:45.272109 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789a855-0ce3-4f25-9678-3b7a68018cd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:45.272823 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a789a855-0ce3-4f25-9678-3b7a68018cd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:45.274470 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a789a855-0ce3-4f25-9678-3b7a68018cd5 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:45.285323 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1160/2305] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:50:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:50:45.289422 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4565ef34-13a7-4b46-970f-d546154e271d admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:45.299356 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4565ef34-13a7-4b46-970f-d546154e271d admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:45.304045 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb4fac-06a5-4221-b5cd-0f12e4ba827d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:45.304924 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb4fac-06a5-4221-b5cd-0f12e4ba827d admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:45.305243 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb4fac-06a5-4221-b5cd-0f12e4ba827d admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:45.305526 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb4fac-06a5-4221-b5cd-0f12e4ba827d admin admin] PATH_INFO: `/v3/projects/3d34eb2980ea4c01bfbcbb2b603bdd9c/users/200c7017b8234060a58b1296c9b1ebf8/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:45.320351 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deb4fac-06a5-4221-b5cd-0f12e4ba827d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d34eb2980ea4c01bfbcbb2b603bdd9c, user_id=200c7017b8234060a58b1296c9b1ebf8, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:45.323054 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deb4fac-06a5-4221-b5cd-0f12e4ba827d admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:45.336106 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1146/2306] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:50:45 2022] PUT /identity/v3/projects/3d34eb2980ea4c01bfbcbb2b603bdd9c/users/200c7017b8234060a58b1296c9b1ebf8/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:50:45.346658 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67ac6a-75bc-4a94-9aff-18f8c55cd4b5 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:45.346951 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67ac6a-75bc-4a94-9aff-18f8c55cd4b5 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:45.347233 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67ac6a-75bc-4a94-9aff-18f8c55cd4b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:45.373423 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-ac67ac6a-75bc-4a94-9aff-18f8c55cd4b5 None None] MFA Rules not processed for user `200c7017b8234060a58b1296c9b1ebf8`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:50:45.374804 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-ac67ac6a-75bc-4a94-9aff-18f8c55cd4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:50:45.439349 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1161/2307] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:50:45 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:50:45.443952 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb4fac-06a5-4221-b5cd-0f12e4ba827d admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:45.507257 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb4fac-06a5-4221-b5cd-0f12e4ba827d admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:45.511882 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1302f8b0-055e-4429-a0af-c92f41da8840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:45.512538 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1302f8b0-055e-4429-a0af-c92f41da8840 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:45.512826 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1302f8b0-055e-4429-a0af-c92f41da8840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:45.513100 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1302f8b0-055e-4429-a0af-c92f41da8840 admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:45.514371 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1302f8b0-055e-4429-a0af-c92f41da8840 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:45.516274 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1302f8b0-055e-4429-a0af-c92f41da8840 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:45.541690 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1147/2308] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:50:45 2022] POST /identity/v3/projects => generated 363 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:50:45.546332 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67ac6a-75bc-4a94-9aff-18f8c55cd4b5 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:45.556914 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67ac6a-75bc-4a94-9aff-18f8c55cd4b5 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:45.612354 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32df6305-1a96-486f-b366-f58735cba778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:45.613023 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df6305-1a96-486f-b366-f58735cba778 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:45.613350 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df6305-1a96-486f-b366-f58735cba778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:45.613657 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df6305-1a96-486f-b366-f58735cba778 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:45.614407 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32df6305-1a96-486f-b366-f58735cba778 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:45.616392 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32df6305-1a96-486f-b366-f58735cba778 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:45.636415 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1162/2309] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:50:45 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:50:45.640995 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1302f8b0-055e-4429-a0af-c92f41da8840 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:45.650974 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1302f8b0-055e-4429-a0af-c92f41da8840 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:45.655547 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730964af-c425-4690-a68e-b0d6ac6386a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:45.656198 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730964af-c425-4690-a68e-b0d6ac6386a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:45.656490 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730964af-c425-4690-a68e-b0d6ac6386a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:45.656771 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730964af-c425-4690-a68e-b0d6ac6386a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:45.657529 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730964af-c425-4690-a68e-b0d6ac6386a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:45.659123 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730964af-c425-4690-a68e-b0d6ac6386a3 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:45.669601 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1148/2310] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:50:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:50:45.673842 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32df6305-1a96-486f-b366-f58735cba778 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:45.684057 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32df6305-1a96-486f-b366-f58735cba778 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:45.689335 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f72c86-8ef8-4fd0-996f-ac19e851b884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:45.690218 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f72c86-8ef8-4fd0-996f-ac19e851b884 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:45.690521 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f72c86-8ef8-4fd0-996f-ac19e851b884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:45.690787 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f72c86-8ef8-4fd0-996f-ac19e851b884 admin admin] PATH_INFO: `/v3/projects/132ef0f9dfc74edba100c6328e744a83/users/1ac7070f569b43e9b2b73504d81edce9/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:45.705901 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f72c86-8ef8-4fd0-996f-ac19e851b884 admin admin] RBAC: Authorizing `identity:create_grant(project_id=132ef0f9dfc74edba100c6328e744a83, user_id=1ac7070f569b43e9b2b73504d81edce9, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:45.708605 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f72c86-8ef8-4fd0-996f-ac19e851b884 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:45.720867 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1163/2311] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:50:45 2022] PUT /identity/v3/projects/132ef0f9dfc74edba100c6328e744a83/users/1ac7070f569b43e9b2b73504d81edce9/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:50:45.724961 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730964af-c425-4690-a68e-b0d6ac6386a3 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:45.734775 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730964af-c425-4690-a68e-b0d6ac6386a3 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:45.791883 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef73870c-da8b-4cb1-9219-bce112396d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:45.792477 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73870c-da8b-4cb1-9219-bce112396d8e admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:45.792763 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73870c-da8b-4cb1-9219-bce112396d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:45.793043 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73870c-da8b-4cb1-9219-bce112396d8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:45.793828 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef73870c-da8b-4cb1-9219-bce112396d8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:45.795439 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef73870c-da8b-4cb1-9219-bce112396d8e admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:45.805283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1149/2312] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:50:45 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:50:45.809540 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f72c86-8ef8-4fd0-996f-ac19e851b884 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:45.820142 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f72c86-8ef8-4fd0-996f-ac19e851b884 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:45.824826 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd566eaf-04a8-4ac9-93c5-f5876c613e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:45.826338 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd566eaf-04a8-4ac9-93c5-f5876c613e79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:45.826637 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd566eaf-04a8-4ac9-93c5-f5876c613e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:45.826986 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd566eaf-04a8-4ac9-93c5-f5876c613e79 admin admin] PATH_INFO: `/v3/projects/132ef0f9dfc74edba100c6328e744a83/users/1ac7070f569b43e9b2b73504d81edce9/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:45.828883 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd566eaf-04a8-4ac9-93c5-f5876c613e79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=132ef0f9dfc74edba100c6328e744a83, user_id=1ac7070f569b43e9b2b73504d81edce9, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:45.831581 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd566eaf-04a8-4ac9-93c5-f5876c613e79 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:45.842036 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1164/2313] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:50:45 2022] PUT /identity/v3/projects/132ef0f9dfc74edba100c6328e744a83/users/1ac7070f569b43e9b2b73504d81edce9/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:50:45.853307 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade276a5-82fc-4fcc-84a2-714a025e0eeb None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:45.853601 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade276a5-82fc-4fcc-84a2-714a025e0eeb None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:45.853882 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade276a5-82fc-4fcc-84a2-714a025e0eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:45.881435 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-ade276a5-82fc-4fcc-84a2-714a025e0eeb None None] MFA Rules not processed for user `1ac7070f569b43e9b2b73504d81edce9`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:50:45.882643 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ade276a5-82fc-4fcc-84a2-714a025e0eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:50:45.892436 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:50:45.952458 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1150/2314] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:50:45 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:50:45.963594 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd566eaf-04a8-4ac9-93c5-f5876c613e79 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:45.965344 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-fd566eaf-04a8-4ac9-93c5-f5876c613e79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:50:45.989102 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:50:46.024768 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd566eaf-04a8-4ac9-93c5-f5876c613e79 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:46.028357 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a696fc-3a45-48c1-9980-f9700612bc3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:46.029173 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a696fc-3a45-48c1-9980-f9700612bc3b service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:46.029464 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a696fc-3a45-48c1-9980-f9700612bc3b service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:46.029743 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a696fc-3a45-48c1-9980-f9700612bc3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:46.038528 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a696fc-3a45-48c1-9980-f9700612bc3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:46.039958 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a696fc-3a45-48c1-9980-f9700612bc3b service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:46.049941 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1165/2315] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:50:45 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:50:46.454341 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade276a5-82fc-4fcc-84a2-714a025e0eeb None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:46.516132 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade276a5-82fc-4fcc-84a2-714a025e0eeb None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:46.520836 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aded49c5-274c-46a8-a1fd-feb826b115b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:46.521747 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aded49c5-274c-46a8-a1fd-feb826b115b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:46.522035 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aded49c5-274c-46a8-a1fd-feb826b115b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:46.522311 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aded49c5-274c-46a8-a1fd-feb826b115b1 admin admin] PATH_INFO: `/v3/users/200c7017b8234060a58b1296c9b1ebf8` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:46.523515 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aded49c5-274c-46a8-a1fd-feb826b115b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=200c7017b8234060a58b1296c9b1ebf8)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:46.525484 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aded49c5-274c-46a8-a1fd-feb826b115b1 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:46.570968 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-aded49c5-274c-46a8-a1fd-feb826b115b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '200c7017b8234060a58b1296c9b1ebf8'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:46.571531 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-aded49c5-274c-46a8-a1fd-feb826b115b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '200c7017b8234060a58b1296c9b1ebf8'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:46.579090 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-aded49c5-274c-46a8-a1fd-feb826b115b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '200c7017b8234060a58b1296c9b1ebf8'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:46.590846 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-aded49c5-274c-46a8-a1fd-feb826b115b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '200c7017b8234060a58b1296c9b1ebf8'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:46.592069 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-aded49c5-274c-46a8-a1fd-feb826b115b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '200c7017b8234060a58b1296c9b1ebf8'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:46.604105 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1151/2316] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:50:46 2022] DELETE /identity/v3/users/200c7017b8234060a58b1296c9b1ebf8 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:50:46.608729 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a7df3-c574-45c9-967d-bfe416a17fc3 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:46.609278 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a7df3-c574-45c9-967d-bfe416a17fc3 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:46.609540 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a7df3-c574-45c9-967d-bfe416a17fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:46.635323 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-b81a7df3-c574-45c9-967d-bfe416a17fc3 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:50:46.636543 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b81a7df3-c574-45c9-967d-bfe416a17fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:50:46.696991 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1166/2317] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:50:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:50:46.707134 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aded49c5-274c-46a8-a1fd-feb826b115b1 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:46.708525 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-aded49c5-274c-46a8-a1fd-feb826b115b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:50:46.739277 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:50:46.768005 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aded49c5-274c-46a8-a1fd-feb826b115b1 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:46.771776 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd40ffd0-3174-468c-b7be-02992028c273 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:46.772446 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40ffd0-3174-468c-b7be-02992028c273 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:46.772735 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40ffd0-3174-468c-b7be-02992028c273 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:46.773018 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40ffd0-3174-468c-b7be-02992028c273 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:46.782744 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd40ffd0-3174-468c-b7be-02992028c273 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:46.784184 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd40ffd0-3174-468c-b7be-02992028c273 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:46.793645 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1152/2318] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:50:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:50:47.067127 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a7df3-c574-45c9-967d-bfe416a17fc3 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:47.068680 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b81a7df3-c574-45c9-967d-bfe416a17fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:50:47.082222 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a7df3-c574-45c9-967d-bfe416a17fc3 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:47.086922 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492f66a3-55d1-44c2-a298-8a3c9efc16b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:47.087785 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f66a3-55d1-44c2-a298-8a3c9efc16b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:47.088076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f66a3-55d1-44c2-a298-8a3c9efc16b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:47.088361 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f66a3-55d1-44c2-a298-8a3c9efc16b0 admin admin] PATH_INFO: `/v3/projects/3d34eb2980ea4c01bfbcbb2b603bdd9c` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:47.089714 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492f66a3-55d1-44c2-a298-8a3c9efc16b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d34eb2980ea4c01bfbcbb2b603bdd9c)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:47.091725 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492f66a3-55d1-44c2-a298-8a3c9efc16b0 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:47.137787 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-492f66a3-55d1-44c2-a298-8a3c9efc16b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d34eb2980ea4c01bfbcbb2b603bdd9c', 'request_id': 'req-492f66a3-55d1-44c2-a298-8a3c9efc16b0'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:47.143714 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-492f66a3-55d1-44c2-a298-8a3c9efc16b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d34eb2980ea4c01bfbcbb2b603bdd9c', 'request_id': 'req-492f66a3-55d1-44c2-a298-8a3c9efc16b0'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:47.149702 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-492f66a3-55d1-44c2-a298-8a3c9efc16b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d34eb2980ea4c01bfbcbb2b603bdd9c', 'request_id': 'req-492f66a3-55d1-44c2-a298-8a3c9efc16b0'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:47.161730 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-492f66a3-55d1-44c2-a298-8a3c9efc16b0 admin admin] The token cache is being invalidate because project 3d34eb2980ea4c01bfbcbb2b603bdd9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:50:47.162017 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-492f66a3-55d1-44c2-a298-8a3c9efc16b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:47.162635 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-492f66a3-55d1-44c2-a298-8a3c9efc16b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:47.164157 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1167/2319] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:50:47 2022] DELETE /identity/v3/projects/3d34eb2980ea4c01bfbcbb2b603bdd9c => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:50:47.168104 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd40ffd0-3174-468c-b7be-02992028c273 service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:47.178067 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd40ffd0-3174-468c-b7be-02992028c273 service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:47.234132 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336039db-6968-4b46-8f39-64cd90d13380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:47.235014 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336039db-6968-4b46-8f39-64cd90d13380 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:47.235299 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336039db-6968-4b46-8f39-64cd90d13380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:47.235575 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336039db-6968-4b46-8f39-64cd90d13380 admin admin] PATH_INFO: `/v3/users/1ac7070f569b43e9b2b73504d81edce9` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:47.236770 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336039db-6968-4b46-8f39-64cd90d13380 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ac7070f569b43e9b2b73504d81edce9)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:47.238739 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336039db-6968-4b46-8f39-64cd90d13380 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:47.285119 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-336039db-6968-4b46-8f39-64cd90d13380 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ac7070f569b43e9b2b73504d81edce9'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:47.285775 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-336039db-6968-4b46-8f39-64cd90d13380 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ac7070f569b43e9b2b73504d81edce9'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:47.292898 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-336039db-6968-4b46-8f39-64cd90d13380 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ac7070f569b43e9b2b73504d81edce9'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:47.304964 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-336039db-6968-4b46-8f39-64cd90d13380 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ac7070f569b43e9b2b73504d81edce9'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:47.305613 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-336039db-6968-4b46-8f39-64cd90d13380 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ac7070f569b43e9b2b73504d81edce9'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:47.318069 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1153/2320] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:50:47 2022] DELETE /identity/v3/users/1ac7070f569b43e9b2b73504d81edce9 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:50:47.580855 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492f66a3-55d1-44c2-a298-8a3c9efc16b0 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:47.582403 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-492f66a3-55d1-44c2-a298-8a3c9efc16b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:50:47.600236 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492f66a3-55d1-44c2-a298-8a3c9efc16b0 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:47.646057 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:50:47.660487 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ab41d-b3a8-442f-98e7-c38e558ebd36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:47.661471 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ab41d-b3a8-442f-98e7-c38e558ebd36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:47.661701 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ab41d-b3a8-442f-98e7-c38e558ebd36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:47.661978 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ab41d-b3a8-442f-98e7-c38e558ebd36 admin admin] PATH_INFO: `/v3/projects/132ef0f9dfc74edba100c6328e744a83` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:47.663197 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2ab41d-b3a8-442f-98e7-c38e558ebd36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=132ef0f9dfc74edba100c6328e744a83)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:47.665117 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2ab41d-b3a8-442f-98e7-c38e558ebd36 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:47.710404 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-2c2ab41d-b3a8-442f-98e7-c38e558ebd36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '132ef0f9dfc74edba100c6328e744a83', 'request_id': 'req-2c2ab41d-b3a8-442f-98e7-c38e558ebd36'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:47.715769 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-2c2ab41d-b3a8-442f-98e7-c38e558ebd36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '132ef0f9dfc74edba100c6328e744a83', 'request_id': 'req-2c2ab41d-b3a8-442f-98e7-c38e558ebd36'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:47.722257 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-2c2ab41d-b3a8-442f-98e7-c38e558ebd36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '132ef0f9dfc74edba100c6328e744a83', 'request_id': 'req-2c2ab41d-b3a8-442f-98e7-c38e558ebd36'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:47.735709 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-2c2ab41d-b3a8-442f-98e7-c38e558ebd36 admin admin] The token cache is being invalidate because project 132ef0f9dfc74edba100c6328e744a83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:50:47.735881 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-2c2ab41d-b3a8-442f-98e7-c38e558ebd36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:47.736544 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-2c2ab41d-b3a8-442f-98e7-c38e558ebd36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:47.738112 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1168/2321] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:50:47 2022] DELETE /identity/v3/projects/132ef0f9dfc74edba100c6328e744a83 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:50:47.743285 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4865f83f-9579-41f5-adb5-b6d3102058f9 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:47.743761 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4865f83f-9579-41f5-adb5-b6d3102058f9 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:47.743951 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4865f83f-9579-41f5-adb5-b6d3102058f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:47.762552 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-4865f83f-9579-41f5-adb5-b6d3102058f9 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:50:47.763793 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-4865f83f-9579-41f5-adb5-b6d3102058f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:50:47.809168 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:50:47.827331 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1154/2322] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:50:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:50:47.835323 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f0b7da-e8b4-4600-8f57-d09dd56a91b4 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:47.835625 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f0b7da-e8b4-4600-8f57-d09dd56a91b4 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:47.835911 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f0b7da-e8b4-4600-8f57-d09dd56a91b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:47.854693 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-e1f0b7da-e8b4-4600-8f57-d09dd56a91b4 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:50:47.855906 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e1f0b7da-e8b4-4600-8f57-d09dd56a91b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:50:47.864546 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1169/2323] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:50:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:50:47.868891 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4865f83f-9579-41f5-adb5-b6d3102058f9 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:47.879341 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4865f83f-9579-41f5-adb5-b6d3102058f9 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:47.884173 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85196b6-8df2-4c18-9328-48098f09a945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:47.884829 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85196b6-8df2-4c18-9328-48098f09a945 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:47.885118 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85196b6-8df2-4c18-9328-48098f09a945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:47.885431 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85196b6-8df2-4c18-9328-48098f09a945 admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:47.886173 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85196b6-8df2-4c18-9328-48098f09a945 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:50:47.886486 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85196b6-8df2-4c18-9328-48098f09a945 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:47.889119 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85196b6-8df2-4c18-9328-48098f09a945 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:47.901270 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1155/2324] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:50:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:50:47.906167 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f0b7da-e8b4-4600-8f57-d09dd56a91b4 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:47.915594 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f0b7da-e8b4-4600-8f57-d09dd56a91b4 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:47.924366 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d6058-a72a-48b9-9315-2fa620e4ceb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:47.925386 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d6058-a72a-48b9-9315-2fa620e4ceb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:47.925922 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d6058-a72a-48b9-9315-2fa620e4ceb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:47.926433 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d6058-a72a-48b9-9315-2fa620e4ceb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:47.927200 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651d6058-a72a-48b9-9315-2fa620e4ceb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:47.929697 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651d6058-a72a-48b9-9315-2fa620e4ceb2 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:47.958662 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1170/2325] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:50:47 2022] POST /identity/v3/projects => generated 363 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:50:47.962874 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85196b6-8df2-4c18-9328-48098f09a945 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:47.973552 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85196b6-8df2-4c18-9328-48098f09a945 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:48.031558 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f1ef3f-dab5-47ce-a2e1-20e8d5aae808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:48.032240 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1ef3f-dab5-47ce-a2e1-20e8d5aae808 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:48.032526 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1ef3f-dab5-47ce-a2e1-20e8d5aae808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:48.032806 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1ef3f-dab5-47ce-a2e1-20e8d5aae808 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:48.033543 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f1ef3f-dab5-47ce-a2e1-20e8d5aae808 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:48.035469 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f1ef3f-dab5-47ce-a2e1-20e8d5aae808 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:48.055779 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1156/2326] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:50:47 2022] POST /identity/v3/users => generated 282 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:50:48.059902 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d6058-a72a-48b9-9315-2fa620e4ceb2 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:48.070250 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d6058-a72a-48b9-9315-2fa620e4ceb2 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:48.075200 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ff5ee-91c7-4513-99ee-059d54664766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:48.076419 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ff5ee-91c7-4513-99ee-059d54664766 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:48.076704 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ff5ee-91c7-4513-99ee-059d54664766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:48.076977 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ff5ee-91c7-4513-99ee-059d54664766 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:48.077746 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59ff5ee-91c7-4513-99ee-059d54664766 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:48.079340 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59ff5ee-91c7-4513-99ee-059d54664766 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:48.089526 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1171/2327] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:50:48.093983 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f1ef3f-dab5-47ce-a2e1-20e8d5aae808 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:48.103692 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f1ef3f-dab5-47ce-a2e1-20e8d5aae808 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:48.108272 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299adb58-86cf-4e1c-af23-9341f0f5ad04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:48.108927 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299adb58-86cf-4e1c-af23-9341f0f5ad04 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:48.109243 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299adb58-86cf-4e1c-af23-9341f0f5ad04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:48.109526 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299adb58-86cf-4e1c-af23-9341f0f5ad04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:48.110253 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299adb58-86cf-4e1c-af23-9341f0f5ad04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:48.111871 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299adb58-86cf-4e1c-af23-9341f0f5ad04 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:48.121879 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1157/2328] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:50:48.125919 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ff5ee-91c7-4513-99ee-059d54664766 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:48.135875 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ff5ee-91c7-4513-99ee-059d54664766 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:48.140930 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0597fb-0fa7-4f1a-bddd-c1d6f093bf99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:48.141846 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0597fb-0fa7-4f1a-bddd-c1d6f093bf99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:48.142132 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0597fb-0fa7-4f1a-bddd-c1d6f093bf99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:48.142407 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0597fb-0fa7-4f1a-bddd-c1d6f093bf99 admin admin] PATH_INFO: `/v3/projects/00a6b984bb8c45dab417ab44f6717d2e/users/bd201b46078a43b6a7ad7bc39a89a925/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:48.157197 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0597fb-0fa7-4f1a-bddd-c1d6f093bf99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00a6b984bb8c45dab417ab44f6717d2e, user_id=bd201b46078a43b6a7ad7bc39a89a925, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:48.159848 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0597fb-0fa7-4f1a-bddd-c1d6f093bf99 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:48.172930 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1172/2329] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:50:48 2022] PUT /identity/v3/projects/00a6b984bb8c45dab417ab44f6717d2e/users/bd201b46078a43b6a7ad7bc39a89a925/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:50:48.183189 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c4cc54-2136-45cd-9882-420aa67182ee None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:48.183530 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c4cc54-2136-45cd-9882-420aa67182ee None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:48.183855 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c4cc54-2136-45cd-9882-420aa67182ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:48.210483 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-79c4cc54-2136-45cd-9882-420aa67182ee None None] MFA Rules not processed for user `bd201b46078a43b6a7ad7bc39a89a925`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:50:48.211696 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-79c4cc54-2136-45cd-9882-420aa67182ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:50:48.276600 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1158/2330] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:50:48 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:50:48.282848 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0597fb-0fa7-4f1a-bddd-c1d6f093bf99 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:48.351857 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0597fb-0fa7-4f1a-bddd-c1d6f093bf99 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:48.356807 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41179111-4a04-4035-9d21-90fa5696f608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:48.357561 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41179111-4a04-4035-9d21-90fa5696f608 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:48.357862 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41179111-4a04-4035-9d21-90fa5696f608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:48.358147 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41179111-4a04-4035-9d21-90fa5696f608 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:48.359779 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41179111-4a04-4035-9d21-90fa5696f608 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:48.361764 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41179111-4a04-4035-9d21-90fa5696f608 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:48.380600 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:50:48.389638 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1173/2331] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:50:48 2022] POST /identity/v3/projects => generated 363 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:50:48.393470 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c4cc54-2136-45cd-9882-420aa67182ee None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:48.404602 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c4cc54-2136-45cd-9882-420aa67182ee None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:48.444203 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:50:48.459822 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbdeb13-8373-4ae7-86c8-e6d499614ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:48.460495 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbdeb13-8373-4ae7-86c8-e6d499614ac7 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:48.460794 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbdeb13-8373-4ae7-86c8-e6d499614ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:48.461058 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbdeb13-8373-4ae7-86c8-e6d499614ac7 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:48.461786 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbdeb13-8373-4ae7-86c8-e6d499614ac7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:48.463707 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbdeb13-8373-4ae7-86c8-e6d499614ac7 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:48.483397 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1159/2332] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:50:48 2022] POST /identity/v3/users => generated 288 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:50:48.487425 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41179111-4a04-4035-9d21-90fa5696f608 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:48.498605 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41179111-4a04-4035-9d21-90fa5696f608 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:48.503401 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79063d38-ca5b-49c5-8fad-5003fc9ff378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:48.504095 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79063d38-ca5b-49c5-8fad-5003fc9ff378 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:48.504383 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79063d38-ca5b-49c5-8fad-5003fc9ff378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:48.504666 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79063d38-ca5b-49c5-8fad-5003fc9ff378 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:48.505445 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79063d38-ca5b-49c5-8fad-5003fc9ff378 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:48.507080 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79063d38-ca5b-49c5-8fad-5003fc9ff378 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:48.518248 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1174/2333] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:50:48.522290 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbdeb13-8373-4ae7-86c8-e6d499614ac7 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:48.533421 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbdeb13-8373-4ae7-86c8-e6d499614ac7 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:48.538206 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ff526-2f0e-45a2-be00-d19fbaed35b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:48.539096 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ff526-2f0e-45a2-be00-d19fbaed35b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:48.539379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ff526-2f0e-45a2-be00-d19fbaed35b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:48.539655 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ff526-2f0e-45a2-be00-d19fbaed35b8 admin admin] PATH_INFO: `/v3/projects/d8215a958e824b05b2a12616fae7a316/users/38312ddefc86410fbb5a41664bf62548/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:48.556458 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748ff526-2f0e-45a2-be00-d19fbaed35b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8215a958e824b05b2a12616fae7a316, user_id=38312ddefc86410fbb5a41664bf62548, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:48.559179 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748ff526-2f0e-45a2-be00-d19fbaed35b8 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:48.572903 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1160/2334] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:50:48 2022] PUT /identity/v3/projects/d8215a958e824b05b2a12616fae7a316/users/38312ddefc86410fbb5a41664bf62548/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:50:48.577547 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79063d38-ca5b-49c5-8fad-5003fc9ff378 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:48.590545 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79063d38-ca5b-49c5-8fad-5003fc9ff378 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:48.647010 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbd71b8-52dc-4e92-9621-7a50c66c7e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:48.647713 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd71b8-52dc-4e92-9621-7a50c66c7e99 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:48.647956 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd71b8-52dc-4e92-9621-7a50c66c7e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:48.648240 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd71b8-52dc-4e92-9621-7a50c66c7e99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:48.648973 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbd71b8-52dc-4e92-9621-7a50c66c7e99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:48.650677 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbd71b8-52dc-4e92-9621-7a50c66c7e99 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:48.660334 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1175/2335] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:50:48.664486 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ff526-2f0e-45a2-be00-d19fbaed35b8 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:48.675021 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ff526-2f0e-45a2-be00-d19fbaed35b8 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:48.679710 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8e14dc-2b93-4058-870e-38131f4109a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:48.680585 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e14dc-2b93-4058-870e-38131f4109a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:48.680867 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e14dc-2b93-4058-870e-38131f4109a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:48.681182 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e14dc-2b93-4058-870e-38131f4109a4 admin admin] PATH_INFO: `/v3/projects/d8215a958e824b05b2a12616fae7a316/users/38312ddefc86410fbb5a41664bf62548/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:48.683144 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8e14dc-2b93-4058-870e-38131f4109a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8215a958e824b05b2a12616fae7a316, user_id=38312ddefc86410fbb5a41664bf62548, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:48.686321 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8e14dc-2b93-4058-870e-38131f4109a4 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:48.696584 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1161/2336] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:50:48 2022] PUT /identity/v3/projects/d8215a958e824b05b2a12616fae7a316/users/38312ddefc86410fbb5a41664bf62548/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:50:48.708235 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad3313-7e26-4787-8fea-4f7e96d2aa20 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:48.708532 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad3313-7e26-4787-8fea-4f7e96d2aa20 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:48.708818 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad3313-7e26-4787-8fea-4f7e96d2aa20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:48.735649 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-81ad3313-7e26-4787-8fea-4f7e96d2aa20 None None] MFA Rules not processed for user `38312ddefc86410fbb5a41664bf62548`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:50:48.736856 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-81ad3313-7e26-4787-8fea-4f7e96d2aa20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:50:48.805702 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1176/2337] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:50:48 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:50:48.817388 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8e14dc-2b93-4058-870e-38131f4109a4 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:48.819090 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-8d8e14dc-2b93-4058-870e-38131f4109a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:50:48.876653 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8e14dc-2b93-4058-870e-38131f4109a4 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:48.880304 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41147a4-f3be-4e54-9b42-ae4dfa32781d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:48.880944 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41147a4-f3be-4e54-9b42-ae4dfa32781d service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:48.881258 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41147a4-f3be-4e54-9b42-ae4dfa32781d service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:48.881538 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41147a4-f3be-4e54-9b42-ae4dfa32781d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:48.890983 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41147a4-f3be-4e54-9b42-ae4dfa32781d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:48.892435 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41147a4-f3be-4e54-9b42-ae4dfa32781d service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:48.902334 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1162/2338] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:50:48 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:50:48.977135 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad3313-7e26-4787-8fea-4f7e96d2aa20 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:48.987228 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad3313-7e26-4787-8fea-4f7e96d2aa20 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:48.990842 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e7520c-04ad-4b26-a3f8-46a5ff02741a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:48.991477 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e7520c-04ad-4b26-a3f8-46a5ff02741a service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:48.991764 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e7520c-04ad-4b26-a3f8-46a5ff02741a service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:48.992083 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e7520c-04ad-4b26-a3f8-46a5ff02741a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:49.048221 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e7520c-04ad-4b26-a3f8-46a5ff02741a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:49.049723 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e7520c-04ad-4b26-a3f8-46a5ff02741a service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:49.058579 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1177/2339] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:50:48 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:50:51.750379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41147a4-f3be-4e54-9b42-ae4dfa32781d service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:51.812276 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41147a4-f3be-4e54-9b42-ae4dfa32781d service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:51.816964 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec18eb-18d2-457c-a956-f26aef45335d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:51.817869 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec18eb-18d2-457c-a956-f26aef45335d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:51.818156 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec18eb-18d2-457c-a956-f26aef45335d admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:51.818437 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec18eb-18d2-457c-a956-f26aef45335d admin admin] PATH_INFO: `/v3/users/bd201b46078a43b6a7ad7bc39a89a925` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:51.819627 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ec18eb-18d2-457c-a956-f26aef45335d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd201b46078a43b6a7ad7bc39a89a925)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:51.821598 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ec18eb-18d2-457c-a956-f26aef45335d admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:51.862529 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:50:51.867823 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-43ec18eb-18d2-457c-a956-f26aef45335d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd201b46078a43b6a7ad7bc39a89a925'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:51.868810 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-43ec18eb-18d2-457c-a956-f26aef45335d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd201b46078a43b6a7ad7bc39a89a925'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:51.876921 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-43ec18eb-18d2-457c-a956-f26aef45335d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd201b46078a43b6a7ad7bc39a89a925'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:51.889020 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-43ec18eb-18d2-457c-a956-f26aef45335d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd201b46078a43b6a7ad7bc39a89a925'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:51.889698 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-43ec18eb-18d2-457c-a956-f26aef45335d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd201b46078a43b6a7ad7bc39a89a925'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:51.902043 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1163/2340] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:50:51 2022] DELETE /identity/v3/users/bd201b46078a43b6a7ad7bc39a89a925 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:50:51.907072 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f20f1-c076-4d7a-9b80-101a106aba2a None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:51.907521 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f20f1-c076-4d7a-9b80-101a106aba2a None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:51.907677 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f20f1-c076-4d7a-9b80-101a106aba2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:51.919689 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:50:51.926855 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-823f20f1-c076-4d7a-9b80-101a106aba2a None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:50:51.928182 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-823f20f1-c076-4d7a-9b80-101a106aba2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:50:51.985591 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1178/2341] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:50:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:50:51.995465 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec18eb-18d2-457c-a956-f26aef45335d admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:51.997301 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-43ec18eb-18d2-457c-a956-f26aef45335d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:50:52.053924 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec18eb-18d2-457c-a956-f26aef45335d admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:52.057663 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10abe092-1ed3-4c89-bf95-b3d701fd4249 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:52.058318 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10abe092-1ed3-4c89-bf95-b3d701fd4249 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:52.058611 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10abe092-1ed3-4c89-bf95-b3d701fd4249 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:52.058897 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10abe092-1ed3-4c89-bf95-b3d701fd4249 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:52.068590 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10abe092-1ed3-4c89-bf95-b3d701fd4249 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:52.070063 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10abe092-1ed3-4c89-bf95-b3d701fd4249 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:52.080250 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1164/2342] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:50:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:50:52.346618 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823f20f1-c076-4d7a-9b80-101a106aba2a None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:52.348075 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-823f20f1-c076-4d7a-9b80-101a106aba2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:50:52.362798 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823f20f1-c076-4d7a-9b80-101a106aba2a None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:52.370023 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a9f80-8435-45bf-9611-71c67d8d6789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:52.370899 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a9f80-8435-45bf-9611-71c67d8d6789 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:52.371290 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a9f80-8435-45bf-9611-71c67d8d6789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:52.371779 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a9f80-8435-45bf-9611-71c67d8d6789 admin admin] PATH_INFO: `/v3/projects/00a6b984bb8c45dab417ab44f6717d2e` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:52.372926 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816a9f80-8435-45bf-9611-71c67d8d6789 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00a6b984bb8c45dab417ab44f6717d2e)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:52.375877 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816a9f80-8435-45bf-9611-71c67d8d6789 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:52.432247 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-816a9f80-8435-45bf-9611-71c67d8d6789 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00a6b984bb8c45dab417ab44f6717d2e', 'request_id': 'req-816a9f80-8435-45bf-9611-71c67d8d6789'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:52.438474 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-816a9f80-8435-45bf-9611-71c67d8d6789 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00a6b984bb8c45dab417ab44f6717d2e', 'request_id': 'req-816a9f80-8435-45bf-9611-71c67d8d6789'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:52.444496 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-816a9f80-8435-45bf-9611-71c67d8d6789 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00a6b984bb8c45dab417ab44f6717d2e', 'request_id': 'req-816a9f80-8435-45bf-9611-71c67d8d6789'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:52.456774 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-816a9f80-8435-45bf-9611-71c67d8d6789 admin admin] The token cache is being invalidate because project 00a6b984bb8c45dab417ab44f6717d2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:50:52.457060 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-816a9f80-8435-45bf-9611-71c67d8d6789 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:52.457702 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-816a9f80-8435-45bf-9611-71c67d8d6789 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:52.459232 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1179/2343] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:50:52 2022] DELETE /identity/v3/projects/00a6b984bb8c45dab417ab44f6717d2e => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:50:52.463281 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10abe092-1ed3-4c89-bf95-b3d701fd4249 service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:52.472877 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10abe092-1ed3-4c89-bf95-b3d701fd4249 service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:52.529950 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18b525-6a5c-4dfc-84b8-772ef867c9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:52.530831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18b525-6a5c-4dfc-84b8-772ef867c9a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:52.531125 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18b525-6a5c-4dfc-84b8-772ef867c9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:52.531410 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18b525-6a5c-4dfc-84b8-772ef867c9a8 admin admin] PATH_INFO: `/v3/users/38312ddefc86410fbb5a41664bf62548` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:52.532616 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c18b525-6a5c-4dfc-84b8-772ef867c9a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38312ddefc86410fbb5a41664bf62548)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:52.534580 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c18b525-6a5c-4dfc-84b8-772ef867c9a8 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:52.579970 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8c18b525-6a5c-4dfc-84b8-772ef867c9a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38312ddefc86410fbb5a41664bf62548'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:52.580623 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8c18b525-6a5c-4dfc-84b8-772ef867c9a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38312ddefc86410fbb5a41664bf62548'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:52.587500 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8c18b525-6a5c-4dfc-84b8-772ef867c9a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38312ddefc86410fbb5a41664bf62548'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:52.595126 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:50:52.603507 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8c18b525-6a5c-4dfc-84b8-772ef867c9a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38312ddefc86410fbb5a41664bf62548'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:52.604448 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8c18b525-6a5c-4dfc-84b8-772ef867c9a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38312ddefc86410fbb5a41664bf62548'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:52.617680 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1165/2344] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:50:52 2022] DELETE /identity/v3/users/38312ddefc86410fbb5a41664bf62548 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:50:52.900449 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a9f80-8435-45bf-9611-71c67d8d6789 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:52.901970 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-816a9f80-8435-45bf-9611-71c67d8d6789 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:50:52.916154 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a9f80-8435-45bf-9611-71c67d8d6789 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:52.976744 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0fb425-96de-419a-aa82-61e0632c5a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:52.977666 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0fb425-96de-419a-aa82-61e0632c5a50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:52.977955 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0fb425-96de-419a-aa82-61e0632c5a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:52.978345 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0fb425-96de-419a-aa82-61e0632c5a50 admin admin] PATH_INFO: `/v3/projects/d8215a958e824b05b2a12616fae7a316` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:52.979562 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0fb425-96de-419a-aa82-61e0632c5a50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8215a958e824b05b2a12616fae7a316)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:52.981669 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0fb425-96de-419a-aa82-61e0632c5a50 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:53.021452 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:50:53.035062 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-da0fb425-96de-419a-aa82-61e0632c5a50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8215a958e824b05b2a12616fae7a316', 'request_id': 'req-da0fb425-96de-419a-aa82-61e0632c5a50'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:53.040469 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-da0fb425-96de-419a-aa82-61e0632c5a50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8215a958e824b05b2a12616fae7a316', 'request_id': 'req-da0fb425-96de-419a-aa82-61e0632c5a50'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:53.046577 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-da0fb425-96de-419a-aa82-61e0632c5a50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8215a958e824b05b2a12616fae7a316', 'request_id': 'req-da0fb425-96de-419a-aa82-61e0632c5a50'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:53.058193 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-da0fb425-96de-419a-aa82-61e0632c5a50 admin admin] The token cache is being invalidate because project d8215a958e824b05b2a12616fae7a316 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:50:53.058480 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-da0fb425-96de-419a-aa82-61e0632c5a50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:53.059104 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-da0fb425-96de-419a-aa82-61e0632c5a50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:53.060634 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1180/2345] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:50:52 2022] DELETE /identity/v3/projects/d8215a958e824b05b2a12616fae7a316 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:50:53.065548 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b2321-5fb0-416c-b889-502c4b7c2f5c None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:53.066078 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b2321-5fb0-416c-b889-502c4b7c2f5c None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:53.066177 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b2321-5fb0-416c-b889-502c4b7c2f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:53.084805 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-093b2321-5fb0-416c-b889-502c4b7c2f5c None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:50:53.086070 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-093b2321-5fb0-416c-b889-502c4b7c2f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:50:53.145198 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1166/2346] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:50:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:50:53.152768 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd752826-066e-46fc-b746-be3139fae20c None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:53.153062 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd752826-066e-46fc-b746-be3139fae20c None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:53.153376 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd752826-066e-46fc-b746-be3139fae20c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:53.172000 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-fd752826-066e-46fc-b746-be3139fae20c None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:50:53.173232 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-fd752826-066e-46fc-b746-be3139fae20c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:50:53.181024 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1181/2347] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:50:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:50:53.185109 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b2321-5fb0-416c-b889-502c4b7c2f5c None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:53.194794 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b2321-5fb0-416c-b889-502c4b7c2f5c None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:53.199305 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf559887-cd58-4341-99e0-8b4f8095b7b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:53.199957 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf559887-cd58-4341-99e0-8b4f8095b7b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:53.200242 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf559887-cd58-4341-99e0-8b4f8095b7b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:53.200518 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf559887-cd58-4341-99e0-8b4f8095b7b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:53.201281 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf559887-cd58-4341-99e0-8b4f8095b7b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:50:53.201616 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf559887-cd58-4341-99e0-8b4f8095b7b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:53.203767 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf559887-cd58-4341-99e0-8b4f8095b7b2 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:53.215475 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1167/2348] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:50:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:50:53.220530 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd752826-066e-46fc-b746-be3139fae20c None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:53.229692 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd752826-066e-46fc-b746-be3139fae20c None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:53.234405 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283156a3-e276-48a4-9808-83332fd32bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:53.235056 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283156a3-e276-48a4-9808-83332fd32bdd admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:53.235341 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283156a3-e276-48a4-9808-83332fd32bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:53.235621 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283156a3-e276-48a4-9808-83332fd32bdd admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:53.236342 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283156a3-e276-48a4-9808-83332fd32bdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:53.238329 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283156a3-e276-48a4-9808-83332fd32bdd admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:53.264891 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1182/2349] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:50:53 2022] POST /identity/v3/projects => generated 369 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:50:53.269089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf559887-cd58-4341-99e0-8b4f8095b7b2 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:53.279103 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf559887-cd58-4341-99e0-8b4f8095b7b2 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:53.336532 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344f4c0f-e58d-4b56-a398-9d32f2961a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:53.337228 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f4c0f-e58d-4b56-a398-9d32f2961a97 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:53.337517 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f4c0f-e58d-4b56-a398-9d32f2961a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:53.337801 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f4c0f-e58d-4b56-a398-9d32f2961a97 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:53.338507 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344f4c0f-e58d-4b56-a398-9d32f2961a97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:53.340436 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344f4c0f-e58d-4b56-a398-9d32f2961a97 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:53.360606 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1168/2350] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:50:53 2022] POST /identity/v3/users => generated 285 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:50:53.364765 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283156a3-e276-48a4-9808-83332fd32bdd admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:53.375665 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283156a3-e276-48a4-9808-83332fd32bdd admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:53.380602 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab910ae-ab31-4f99-84ce-87830d754c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:53.381289 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab910ae-ab31-4f99-84ce-87830d754c61 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:53.381576 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab910ae-ab31-4f99-84ce-87830d754c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:53.381854 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab910ae-ab31-4f99-84ce-87830d754c61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:53.382670 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab910ae-ab31-4f99-84ce-87830d754c61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:53.384276 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab910ae-ab31-4f99-84ce-87830d754c61 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:53.394987 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1183/2351] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:50:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:50:53.398950 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344f4c0f-e58d-4b56-a398-9d32f2961a97 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:53.408806 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344f4c0f-e58d-4b56-a398-9d32f2961a97 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:53.413455 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ae7664-7396-4505-ba83-4788c86c107e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:53.414198 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae7664-7396-4505-ba83-4788c86c107e admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:53.414497 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae7664-7396-4505-ba83-4788c86c107e admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:53.414767 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae7664-7396-4505-ba83-4788c86c107e admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:53.415503 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ae7664-7396-4505-ba83-4788c86c107e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:53.417101 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ae7664-7396-4505-ba83-4788c86c107e admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:53.426904 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1169/2352] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:50:53 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:50:53.430837 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab910ae-ab31-4f99-84ce-87830d754c61 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:53.442170 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab910ae-ab31-4f99-84ce-87830d754c61 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:53.447051 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95650d01-b103-4573-b468-b99b101c1746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:53.447881 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95650d01-b103-4573-b468-b99b101c1746 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:53.448163 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95650d01-b103-4573-b468-b99b101c1746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:53.448488 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95650d01-b103-4573-b468-b99b101c1746 admin admin] PATH_INFO: `/v3/projects/c3be3f0be08b470c8b270603e835add4/users/d6fd0aaa309d457b9c8c84fb4185e754/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:53.466650 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95650d01-b103-4573-b468-b99b101c1746 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3be3f0be08b470c8b270603e835add4, user_id=d6fd0aaa309d457b9c8c84fb4185e754, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:53.469406 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95650d01-b103-4573-b468-b99b101c1746 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:53.481997 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1184/2353] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:50:53 2022] PUT /identity/v3/projects/c3be3f0be08b470c8b270603e835add4/users/d6fd0aaa309d457b9c8c84fb4185e754/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:50:53.493471 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb12350-58ce-4c03-ada4-736add10c7be None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:53.493773 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb12350-58ce-4c03-ada4-736add10c7be None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:53.494114 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb12350-58ce-4c03-ada4-736add10c7be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:53.521474 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-cdb12350-58ce-4c03-ada4-736add10c7be None None] MFA Rules not processed for user `d6fd0aaa309d457b9c8c84fb4185e754`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:50:53.522694 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-cdb12350-58ce-4c03-ada4-736add10c7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:50:53.554595 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:50:53.586989 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1170/2354] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:50:53 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:50:53.595577 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95650d01-b103-4573-b468-b99b101c1746 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:53.657823 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95650d01-b103-4573-b468-b99b101c1746 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:53.662575 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c373a2-af5d-4bdb-8354-baca19890b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:53.663232 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c373a2-af5d-4bdb-8354-baca19890b82 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:53.663520 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c373a2-af5d-4bdb-8354-baca19890b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:53.663802 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c373a2-af5d-4bdb-8354-baca19890b82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:53.664492 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c373a2-af5d-4bdb-8354-baca19890b82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:53.666445 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c373a2-af5d-4bdb-8354-baca19890b82 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:53.691473 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1185/2355] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:50:53 2022] POST /identity/v3/projects => generated 367 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:50:53.695649 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb12350-58ce-4c03-ada4-736add10c7be None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:53.707158 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb12350-58ce-4c03-ada4-736add10c7be None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:53.762615 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c75ad46-8a87-45bb-a99f-308533dfe1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:53.763286 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75ad46-8a87-45bb-a99f-308533dfe1a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:53.763573 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75ad46-8a87-45bb-a99f-308533dfe1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:53.763852 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75ad46-8a87-45bb-a99f-308533dfe1a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:53.764584 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c75ad46-8a87-45bb-a99f-308533dfe1a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:53.766553 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c75ad46-8a87-45bb-a99f-308533dfe1a6 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:53.786583 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1171/2356] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:50:53 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:50:53.790629 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c373a2-af5d-4bdb-8354-baca19890b82 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:53.801276 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c373a2-af5d-4bdb-8354-baca19890b82 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:53.805867 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d099ad2-d23c-4f2f-af63-92c711fb0b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:53.806519 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d099ad2-d23c-4f2f-af63-92c711fb0b30 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:53.806805 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d099ad2-d23c-4f2f-af63-92c711fb0b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:53.807082 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d099ad2-d23c-4f2f-af63-92c711fb0b30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:53.807814 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d099ad2-d23c-4f2f-af63-92c711fb0b30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:53.809451 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d099ad2-d23c-4f2f-af63-92c711fb0b30 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:53.821366 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1186/2357] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:50:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:50:53.825508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c75ad46-8a87-45bb-a99f-308533dfe1a6 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:53.835422 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c75ad46-8a87-45bb-a99f-308533dfe1a6 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:53.840079 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d429dfe-3aec-4460-830c-5ce6473089e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:53.840963 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d429dfe-3aec-4460-830c-5ce6473089e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:53.841290 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d429dfe-3aec-4460-830c-5ce6473089e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:53.841576 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d429dfe-3aec-4460-830c-5ce6473089e3 admin admin] PATH_INFO: `/v3/projects/a5efd3ee83f142188b7a55fe5c7e0010/users/b34beb453e074ff9834b458e34337f51/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:53.856147 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d429dfe-3aec-4460-830c-5ce6473089e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5efd3ee83f142188b7a55fe5c7e0010, user_id=b34beb453e074ff9834b458e34337f51, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:53.859372 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d429dfe-3aec-4460-830c-5ce6473089e3 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:53.873313 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1172/2358] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:50:53 2022] PUT /identity/v3/projects/a5efd3ee83f142188b7a55fe5c7e0010/users/b34beb453e074ff9834b458e34337f51/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:50:53.878117 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d099ad2-d23c-4f2f-af63-92c711fb0b30 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:53.888609 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d099ad2-d23c-4f2f-af63-92c711fb0b30 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:53.924287 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:50:53.961618 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed81626-c698-47d3-af65-654c19676b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:53.962201 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed81626-c698-47d3-af65-654c19676b5f admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:53.962489 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed81626-c698-47d3-af65-654c19676b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:53.962770 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed81626-c698-47d3-af65-654c19676b5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:53.963509 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed81626-c698-47d3-af65-654c19676b5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:53.965145 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed81626-c698-47d3-af65-654c19676b5f admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:53.975152 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1187/2359] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:50:53 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:50:53.979391 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d429dfe-3aec-4460-830c-5ce6473089e3 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:53.990446 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d429dfe-3aec-4460-830c-5ce6473089e3 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:53.995106 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e953c0-1031-47a4-941a-0ebd40362bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:53.996017 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e953c0-1031-47a4-941a-0ebd40362bf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:53.996327 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e953c0-1031-47a4-941a-0ebd40362bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:53.996869 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e953c0-1031-47a4-941a-0ebd40362bf9 admin admin] PATH_INFO: `/v3/projects/a5efd3ee83f142188b7a55fe5c7e0010/users/b34beb453e074ff9834b458e34337f51/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:53.999499 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e953c0-1031-47a4-941a-0ebd40362bf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5efd3ee83f142188b7a55fe5c7e0010, user_id=b34beb453e074ff9834b458e34337f51, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:54.002187 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e953c0-1031-47a4-941a-0ebd40362bf9 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:54.012345 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1173/2360] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:50:53 2022] PUT /identity/v3/projects/a5efd3ee83f142188b7a55fe5c7e0010/users/b34beb453e074ff9834b458e34337f51/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:50:54.022658 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2940d2-171e-4312-9b31-9edb7235871f None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:54.022957 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2940d2-171e-4312-9b31-9edb7235871f None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:54.023242 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2940d2-171e-4312-9b31-9edb7235871f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:54.049314 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-2e2940d2-171e-4312-9b31-9edb7235871f None None] MFA Rules not processed for user `b34beb453e074ff9834b458e34337f51`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:50:54.050521 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-2e2940d2-171e-4312-9b31-9edb7235871f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:50:54.118206 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1188/2361] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:50:54 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:50:54.129332 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e953c0-1031-47a4-941a-0ebd40362bf9 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:54.131058 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-26e953c0-1031-47a4-941a-0ebd40362bf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:50:54.188331 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e953c0-1031-47a4-941a-0ebd40362bf9 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:54.191953 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81257fb3-71f2-4496-95ad-eb396f93e573 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:54.192592 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81257fb3-71f2-4496-95ad-eb396f93e573 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:54.192877 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81257fb3-71f2-4496-95ad-eb396f93e573 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:54.193174 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81257fb3-71f2-4496-95ad-eb396f93e573 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:54.202219 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81257fb3-71f2-4496-95ad-eb396f93e573 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:54.203662 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81257fb3-71f2-4496-95ad-eb396f93e573 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:54.213041 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1174/2362] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:50:54 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:50:55.544343 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2940d2-171e-4312-9b31-9edb7235871f None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:55.605424 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2940d2-171e-4312-9b31-9edb7235871f None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:55.610753 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa595e2-426b-4e1b-98a7-f3d50a18082e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:55.611631 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa595e2-426b-4e1b-98a7-f3d50a18082e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:55.611913 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa595e2-426b-4e1b-98a7-f3d50a18082e admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:55.612189 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa595e2-426b-4e1b-98a7-f3d50a18082e admin admin] PATH_INFO: `/v3/users/d6fd0aaa309d457b9c8c84fb4185e754` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:55.613442 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa595e2-426b-4e1b-98a7-f3d50a18082e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6fd0aaa309d457b9c8c84fb4185e754)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:55.615363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa595e2-426b-4e1b-98a7-f3d50a18082e admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:55.673078 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-baa595e2-426b-4e1b-98a7-f3d50a18082e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6fd0aaa309d457b9c8c84fb4185e754'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:55.674034 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-baa595e2-426b-4e1b-98a7-f3d50a18082e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6fd0aaa309d457b9c8c84fb4185e754'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:55.674884 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-baa595e2-426b-4e1b-98a7-f3d50a18082e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6fd0aaa309d457b9c8c84fb4185e754'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:55.684088 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-baa595e2-426b-4e1b-98a7-f3d50a18082e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6fd0aaa309d457b9c8c84fb4185e754'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:55.685333 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:50:55.701583 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-baa595e2-426b-4e1b-98a7-f3d50a18082e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6fd0aaa309d457b9c8c84fb4185e754'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:55.714402 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1189/2363] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:50:55 2022] DELETE /identity/v3/users/d6fd0aaa309d457b9c8c84fb4185e754 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:50:55.718994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e62be9-0311-4894-83b8-98f9fce5cda8 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:55.719456 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e62be9-0311-4894-83b8-98f9fce5cda8 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:55.719558 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e62be9-0311-4894-83b8-98f9fce5cda8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:55.737564 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-96e62be9-0311-4894-83b8-98f9fce5cda8 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:50:55.738822 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-96e62be9-0311-4894-83b8-98f9fce5cda8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:50:55.766243 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:50:55.800075 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1175/2364] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:50:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:50:55.810088 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa595e2-426b-4e1b-98a7-f3d50a18082e admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:55.811979 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-baa595e2-426b-4e1b-98a7-f3d50a18082e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:50:55.867990 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa595e2-426b-4e1b-98a7-f3d50a18082e admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:55.871880 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0516fd07-aff5-4655-8cf1-c49475ce599b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:55.872518 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0516fd07-aff5-4655-8cf1-c49475ce599b service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:55.872801 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0516fd07-aff5-4655-8cf1-c49475ce599b service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:55.873075 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0516fd07-aff5-4655-8cf1-c49475ce599b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:55.882714 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0516fd07-aff5-4655-8cf1-c49475ce599b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:55.884163 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0516fd07-aff5-4655-8cf1-c49475ce599b service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:55.894586 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1190/2365] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:50:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:50:56.171604 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e62be9-0311-4894-83b8-98f9fce5cda8 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:56.173316 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-96e62be9-0311-4894-83b8-98f9fce5cda8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:50:56.187379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e62be9-0311-4894-83b8-98f9fce5cda8 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:56.192311 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa91704-51e3-4129-ab9e-9cc212e34ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:56.193242 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa91704-51e3-4129-ab9e-9cc212e34ae3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:56.193534 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa91704-51e3-4129-ab9e-9cc212e34ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:56.193813 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa91704-51e3-4129-ab9e-9cc212e34ae3 admin admin] PATH_INFO: `/v3/projects/c3be3f0be08b470c8b270603e835add4` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:56.195083 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa91704-51e3-4129-ab9e-9cc212e34ae3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3be3f0be08b470c8b270603e835add4)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:56.197293 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa91704-51e3-4129-ab9e-9cc212e34ae3 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:56.244390 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-bfa91704-51e3-4129-ab9e-9cc212e34ae3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3be3f0be08b470c8b270603e835add4', 'request_id': 'req-bfa91704-51e3-4129-ab9e-9cc212e34ae3'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:56.250609 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-bfa91704-51e3-4129-ab9e-9cc212e34ae3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3be3f0be08b470c8b270603e835add4', 'request_id': 'req-bfa91704-51e3-4129-ab9e-9cc212e34ae3'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:56.256645 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-bfa91704-51e3-4129-ab9e-9cc212e34ae3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3be3f0be08b470c8b270603e835add4', 'request_id': 'req-bfa91704-51e3-4129-ab9e-9cc212e34ae3'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:56.268890 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-bfa91704-51e3-4129-ab9e-9cc212e34ae3 admin admin] The token cache is being invalidate because project c3be3f0be08b470c8b270603e835add4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:50:56.269197 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-bfa91704-51e3-4129-ab9e-9cc212e34ae3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:56.269835 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-bfa91704-51e3-4129-ab9e-9cc212e34ae3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:56.271377 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1176/2366] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:50:56 2022] DELETE /identity/v3/projects/c3be3f0be08b470c8b270603e835add4 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:50:56.275374 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0516fd07-aff5-4655-8cf1-c49475ce599b service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:56.285051 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0516fd07-aff5-4655-8cf1-c49475ce599b service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:56.339943 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e10d4fc-2af3-4648-bd86-2e0ed6afa24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:56.340733 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10d4fc-2af3-4648-bd86-2e0ed6afa24b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:56.341015 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10d4fc-2af3-4648-bd86-2e0ed6afa24b admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:56.341322 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10d4fc-2af3-4648-bd86-2e0ed6afa24b admin admin] PATH_INFO: `/v3/users/b34beb453e074ff9834b458e34337f51` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:56.343041 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e10d4fc-2af3-4648-bd86-2e0ed6afa24b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b34beb453e074ff9834b458e34337f51)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:56.344969 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e10d4fc-2af3-4648-bd86-2e0ed6afa24b admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:56.393675 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-1e10d4fc-2af3-4648-bd86-2e0ed6afa24b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b34beb453e074ff9834b458e34337f51'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:56.394384 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-1e10d4fc-2af3-4648-bd86-2e0ed6afa24b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b34beb453e074ff9834b458e34337f51'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:56.395035 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-1e10d4fc-2af3-4648-bd86-2e0ed6afa24b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b34beb453e074ff9834b458e34337f51'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:56.402476 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-1e10d4fc-2af3-4648-bd86-2e0ed6afa24b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b34beb453e074ff9834b458e34337f51'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:56.415337 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-1e10d4fc-2af3-4648-bd86-2e0ed6afa24b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b34beb453e074ff9834b458e34337f51'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:56.421310 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:50:56.432429 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1191/2367] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:50:56 2022] DELETE /identity/v3/users/b34beb453e074ff9834b458e34337f51 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:50:56.698768 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa91704-51e3-4129-ab9e-9cc212e34ae3 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:56.700224 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-bfa91704-51e3-4129-ab9e-9cc212e34ae3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:50:56.715461 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa91704-51e3-4129-ab9e-9cc212e34ae3 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:56.773079 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fefbd4d-2fcd-4e3f-b1c9-af18b1a42b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:56.773989 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fefbd4d-2fcd-4e3f-b1c9-af18b1a42b45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:56.774275 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fefbd4d-2fcd-4e3f-b1c9-af18b1a42b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:56.774554 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fefbd4d-2fcd-4e3f-b1c9-af18b1a42b45 admin admin] PATH_INFO: `/v3/projects/a5efd3ee83f142188b7a55fe5c7e0010` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:56.775738 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fefbd4d-2fcd-4e3f-b1c9-af18b1a42b45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5efd3ee83f142188b7a55fe5c7e0010)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:56.777709 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fefbd4d-2fcd-4e3f-b1c9-af18b1a42b45 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:56.823017 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0fefbd4d-2fcd-4e3f-b1c9-af18b1a42b45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5efd3ee83f142188b7a55fe5c7e0010', 'request_id': 'req-0fefbd4d-2fcd-4e3f-b1c9-af18b1a42b45'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:56.827784 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:50:56.830228 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0fefbd4d-2fcd-4e3f-b1c9-af18b1a42b45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5efd3ee83f142188b7a55fe5c7e0010', 'request_id': 'req-0fefbd4d-2fcd-4e3f-b1c9-af18b1a42b45'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:56.838257 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0fefbd4d-2fcd-4e3f-b1c9-af18b1a42b45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5efd3ee83f142188b7a55fe5c7e0010', 'request_id': 'req-0fefbd4d-2fcd-4e3f-b1c9-af18b1a42b45'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:56.852957 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0fefbd4d-2fcd-4e3f-b1c9-af18b1a42b45 admin admin] The token cache is being invalidate because project a5efd3ee83f142188b7a55fe5c7e0010 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:50:56.853293 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0fefbd4d-2fcd-4e3f-b1c9-af18b1a42b45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:56.853909 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0fefbd4d-2fcd-4e3f-b1c9-af18b1a42b45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:50:56.855504 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1177/2368] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:50:56 2022] DELETE /identity/v3/projects/a5efd3ee83f142188b7a55fe5c7e0010 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:50:56.860539 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79645484-8137-430e-9c29-b482563aca9f None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:56.861027 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79645484-8137-430e-9c29-b482563aca9f None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:56.861143 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79645484-8137-430e-9c29-b482563aca9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:56.879935 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-79645484-8137-430e-9c29-b482563aca9f None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:50:56.881072 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-79645484-8137-430e-9c29-b482563aca9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:50:56.942205 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1192/2369] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:50:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:50:56.950011 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d1c9dd-bc8e-4be9-8d4b-8a805fc74ede None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:56.950304 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d1c9dd-bc8e-4be9-8d4b-8a805fc74ede None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:56.950610 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d1c9dd-bc8e-4be9-8d4b-8a805fc74ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:56.970594 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-18d1c9dd-bc8e-4be9-8d4b-8a805fc74ede None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:50:56.971790 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-18d1c9dd-bc8e-4be9-8d4b-8a805fc74ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:50:56.979668 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1178/2370] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:50:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:50:56.983962 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79645484-8137-430e-9c29-b482563aca9f None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:56.994275 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79645484-8137-430e-9c29-b482563aca9f None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:57.001467 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a96f668-4953-4e23-bbe3-594c03628f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:57.002142 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a96f668-4953-4e23-bbe3-594c03628f41 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:57.002431 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a96f668-4953-4e23-bbe3-594c03628f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:57.002716 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a96f668-4953-4e23-bbe3-594c03628f41 admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:57.003452 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a96f668-4953-4e23-bbe3-594c03628f41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:50:57.003768 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a96f668-4953-4e23-bbe3-594c03628f41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:57.005440 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a96f668-4953-4e23-bbe3-594c03628f41 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:57.018082 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1193/2371] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:50:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:50:57.022466 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d1c9dd-bc8e-4be9-8d4b-8a805fc74ede None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:57.032271 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d1c9dd-bc8e-4be9-8d4b-8a805fc74ede None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:57.036952 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df98a08-7c9a-49c0-b833-b15bb0a6fa9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:57.037648 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df98a08-7c9a-49c0-b833-b15bb0a6fa9a admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:57.037943 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df98a08-7c9a-49c0-b833-b15bb0a6fa9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:57.038275 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df98a08-7c9a-49c0-b833-b15bb0a6fa9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:57.038965 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df98a08-7c9a-49c0-b833-b15bb0a6fa9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:57.040922 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df98a08-7c9a-49c0-b833-b15bb0a6fa9a admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:57.065873 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1179/2372] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:50:57 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:50:57.070009 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a96f668-4953-4e23-bbe3-594c03628f41 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:57.080746 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a96f668-4953-4e23-bbe3-594c03628f41 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:57.137687 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1256160d-da5d-4363-b4da-16c939d698a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:57.138277 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256160d-da5d-4363-b4da-16c939d698a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:57.138565 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256160d-da5d-4363-b4da-16c939d698a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:57.138843 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256160d-da5d-4363-b4da-16c939d698a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:57.139575 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1256160d-da5d-4363-b4da-16c939d698a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:57.142149 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1256160d-da5d-4363-b4da-16c939d698a6 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:57.162006 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1194/2373] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:50:57 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:50:57.166264 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df98a08-7c9a-49c0-b833-b15bb0a6fa9a admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:57.176458 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df98a08-7c9a-49c0-b833-b15bb0a6fa9a admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:57.181035 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88979acb-1b8d-48b6-a800-f1c79f40bc55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:57.181724 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88979acb-1b8d-48b6-a800-f1c79f40bc55 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:57.182009 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88979acb-1b8d-48b6-a800-f1c79f40bc55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:57.182293 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88979acb-1b8d-48b6-a800-f1c79f40bc55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:57.183024 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88979acb-1b8d-48b6-a800-f1c79f40bc55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:57.184905 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88979acb-1b8d-48b6-a800-f1c79f40bc55 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:57.195674 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1180/2374] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:50:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:50:57.199641 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1256160d-da5d-4363-b4da-16c939d698a6 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:57.210475 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1256160d-da5d-4363-b4da-16c939d698a6 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:57.215152 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2011ae62-2c21-4d17-a91a-22a4211c2285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:57.215814 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2011ae62-2c21-4d17-a91a-22a4211c2285 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:57.216105 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2011ae62-2c21-4d17-a91a-22a4211c2285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:57.216388 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2011ae62-2c21-4d17-a91a-22a4211c2285 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:57.217123 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2011ae62-2c21-4d17-a91a-22a4211c2285 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:57.218809 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2011ae62-2c21-4d17-a91a-22a4211c2285 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:57.231166 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1195/2375] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:50:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:50:57.235236 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88979acb-1b8d-48b6-a800-f1c79f40bc55 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:57.245131 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88979acb-1b8d-48b6-a800-f1c79f40bc55 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:57.250448 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b00c89d-112c-4c49-bc55-872d8ceb7a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:57.251326 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b00c89d-112c-4c49-bc55-872d8ceb7a51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:57.251603 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b00c89d-112c-4c49-bc55-872d8ceb7a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:57.251878 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b00c89d-112c-4c49-bc55-872d8ceb7a51 admin admin] PATH_INFO: `/v3/projects/830095fbe2d64d22bf132086d33c7f36/users/3f0ec213977b4f41b7c3c6af26a3fc79/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:57.266678 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b00c89d-112c-4c49-bc55-872d8ceb7a51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=830095fbe2d64d22bf132086d33c7f36, user_id=3f0ec213977b4f41b7c3c6af26a3fc79, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:57.269349 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b00c89d-112c-4c49-bc55-872d8ceb7a51 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:57.283475 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1181/2376] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:50:57 2022] PUT /identity/v3/projects/830095fbe2d64d22bf132086d33c7f36/users/3f0ec213977b4f41b7c3c6af26a3fc79/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:50:57.294623 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14c0a1-37a9-4702-8816-0643f1136fde None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:57.294993 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14c0a1-37a9-4702-8816-0643f1136fde None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:57.295280 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14c0a1-37a9-4702-8816-0643f1136fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:57.321391 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-9f14c0a1-37a9-4702-8816-0643f1136fde None None] MFA Rules not processed for user `3f0ec213977b4f41b7c3c6af26a3fc79`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:50:57.322776 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-9f14c0a1-37a9-4702-8816-0643f1136fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:50:57.369522 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:50:57.388588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1196/2377] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:50:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:50:57.393949 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b00c89d-112c-4c49-bc55-872d8ceb7a51 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:57.464760 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b00c89d-112c-4c49-bc55-872d8ceb7a51 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:57.469595 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5fc616-ffa2-4340-851d-ba1666026d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:57.470260 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5fc616-ffa2-4340-851d-ba1666026d96 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:57.470551 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5fc616-ffa2-4340-851d-ba1666026d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:57.470832 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5fc616-ffa2-4340-851d-ba1666026d96 admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:57.471553 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5fc616-ffa2-4340-851d-ba1666026d96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:57.473570 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5fc616-ffa2-4340-851d-ba1666026d96 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:57.499664 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1182/2378] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:50:57 2022] POST /identity/v3/projects => generated 387 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:50:57.505079 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f14c0a1-37a9-4702-8816-0643f1136fde None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:57.515757 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f14c0a1-37a9-4702-8816-0643f1136fde None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:57.570903 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eacd45-2bc6-4f80-b6f4-77af23ad97bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:57.571576 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eacd45-2bc6-4f80-b6f4-77af23ad97bd admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:57.571856 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eacd45-2bc6-4f80-b6f4-77af23ad97bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:57.572133 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eacd45-2bc6-4f80-b6f4-77af23ad97bd admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:57.572818 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eacd45-2bc6-4f80-b6f4-77af23ad97bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:57.574797 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eacd45-2bc6-4f80-b6f4-77af23ad97bd admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:57.594129 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1197/2379] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:50:57 2022] POST /identity/v3/users => generated 294 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:50:57.598475 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5fc616-ffa2-4340-851d-ba1666026d96 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:57.608791 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5fc616-ffa2-4340-851d-ba1666026d96 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:57.614015 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da13ab7d-5b97-4a63-aadb-bd6f4a98f41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:57.614677 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13ab7d-5b97-4a63-aadb-bd6f4a98f41c admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:57.614964 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13ab7d-5b97-4a63-aadb-bd6f4a98f41c admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:57.615247 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13ab7d-5b97-4a63-aadb-bd6f4a98f41c admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:57.615973 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da13ab7d-5b97-4a63-aadb-bd6f4a98f41c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:57.617612 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da13ab7d-5b97-4a63-aadb-bd6f4a98f41c admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:57.628927 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1183/2380] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:50:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:50:57.632856 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eacd45-2bc6-4f80-b6f4-77af23ad97bd admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:57.642853 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eacd45-2bc6-4f80-b6f4-77af23ad97bd admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:57.647336 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e151466-0a08-4ed5-90e0-37476ef51a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:57.647987 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e151466-0a08-4ed5-90e0-37476ef51a36 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:57.648270 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e151466-0a08-4ed5-90e0-37476ef51a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:57.648547 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e151466-0a08-4ed5-90e0-37476ef51a36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:57.649307 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e151466-0a08-4ed5-90e0-37476ef51a36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:57.650909 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e151466-0a08-4ed5-90e0-37476ef51a36 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:57.660655 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1198/2381] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:50:57 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:50:57.664619 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da13ab7d-5b97-4a63-aadb-bd6f4a98f41c admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:57.674937 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da13ab7d-5b97-4a63-aadb-bd6f4a98f41c admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:57.679499 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d8829a-ae73-4da1-988b-076d2fe17b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:57.680370 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8829a-ae73-4da1-988b-076d2fe17b52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:57.680660 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8829a-ae73-4da1-988b-076d2fe17b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:57.680944 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8829a-ae73-4da1-988b-076d2fe17b52 admin admin] PATH_INFO: `/v3/projects/92d6e4ad3c6b40bea5059dd027bc8a0f/users/2031d822edc8407bb7ffbc0be2fb52ad/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:57.696475 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d8829a-ae73-4da1-988b-076d2fe17b52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92d6e4ad3c6b40bea5059dd027bc8a0f, user_id=2031d822edc8407bb7ffbc0be2fb52ad, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:57.699167 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d8829a-ae73-4da1-988b-076d2fe17b52 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:57.711951 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1184/2382] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:50:57 2022] PUT /identity/v3/projects/92d6e4ad3c6b40bea5059dd027bc8a0f/users/2031d822edc8407bb7ffbc0be2fb52ad/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:50:57.722091 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd9677-f2b3-417c-85cb-4bce49a8ddff None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:57.722389 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd9677-f2b3-417c-85cb-4bce49a8ddff None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:57.722674 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd9677-f2b3-417c-85cb-4bce49a8ddff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:57.749693 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-ebbd9677-f2b3-417c-85cb-4bce49a8ddff None None] MFA Rules not processed for user `2031d822edc8407bb7ffbc0be2fb52ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:50:57.751464 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-ebbd9677-f2b3-417c-85cb-4bce49a8ddff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:50:57.815263 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1199/2383] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:50:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:50:57.819786 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d8829a-ae73-4da1-988b-076d2fe17b52 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:57.853177 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:50:57.883757 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d8829a-ae73-4da1-988b-076d2fe17b52 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:57.888509 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67fc8a3-0251-4ce9-b0d5-9e8836a5e7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:57.889242 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67fc8a3-0251-4ce9-b0d5-9e8836a5e7d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:57.889538 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67fc8a3-0251-4ce9-b0d5-9e8836a5e7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:57.889824 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67fc8a3-0251-4ce9-b0d5-9e8836a5e7d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:57.890542 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67fc8a3-0251-4ce9-b0d5-9e8836a5e7d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:57.892551 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67fc8a3-0251-4ce9-b0d5-9e8836a5e7d4 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:57.918326 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1185/2384] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:50:57 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:50:57.923306 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd9677-f2b3-417c-85cb-4bce49a8ddff None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:57.934244 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd9677-f2b3-417c-85cb-4bce49a8ddff None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:57.985951 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:50:57.992736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b99f78-913c-44ef-8498-e0cf9f69786c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:57.993458 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b99f78-913c-44ef-8498-e0cf9f69786c admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:57.993939 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b99f78-913c-44ef-8498-e0cf9f69786c admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:57.994475 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b99f78-913c-44ef-8498-e0cf9f69786c admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:57.995292 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b99f78-913c-44ef-8498-e0cf9f69786c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:57.997699 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b99f78-913c-44ef-8498-e0cf9f69786c admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:58.016617 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1200/2385] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:50:57 2022] POST /identity/v3/users => generated 299 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:50:58.020675 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67fc8a3-0251-4ce9-b0d5-9e8836a5e7d4 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:58.030995 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67fc8a3-0251-4ce9-b0d5-9e8836a5e7d4 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:58.036307 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69983c82-ca03-4e29-952f-12bf742e7301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:58.036954 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69983c82-ca03-4e29-952f-12bf742e7301 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:58.037277 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69983c82-ca03-4e29-952f-12bf742e7301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:58.037558 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69983c82-ca03-4e29-952f-12bf742e7301 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:58.038278 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69983c82-ca03-4e29-952f-12bf742e7301 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:58.039870 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69983c82-ca03-4e29-952f-12bf742e7301 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:58.050844 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1186/2386] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:50:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:50:58.055062 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b99f78-913c-44ef-8498-e0cf9f69786c admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:58.064883 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b99f78-913c-44ef-8498-e0cf9f69786c admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:58.069544 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806a8612-f188-47f4-9b89-34e7293162d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:58.070417 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a8612-f188-47f4-9b89-34e7293162d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:58.070706 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a8612-f188-47f4-9b89-34e7293162d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:58.070991 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a8612-f188-47f4-9b89-34e7293162d1 admin admin] PATH_INFO: `/v3/projects/21306330a45642d6855703b602e674df/users/fbb1af30f007422eaadbb44c32cdcba5/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:58.085498 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806a8612-f188-47f4-9b89-34e7293162d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21306330a45642d6855703b602e674df, user_id=fbb1af30f007422eaadbb44c32cdcba5, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:58.088199 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806a8612-f188-47f4-9b89-34e7293162d1 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:58.101593 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1201/2387] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:50:58 2022] PUT /identity/v3/projects/21306330a45642d6855703b602e674df/users/fbb1af30f007422eaadbb44c32cdcba5/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:50:58.105476 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69983c82-ca03-4e29-952f-12bf742e7301 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:58.115935 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69983c82-ca03-4e29-952f-12bf742e7301 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:58.170321 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacd5c2a-b211-463e-bc39-799da9b88d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:58.170904 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd5c2a-b211-463e-bc39-799da9b88d1a admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:58.171191 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd5c2a-b211-463e-bc39-799da9b88d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:58.171472 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd5c2a-b211-463e-bc39-799da9b88d1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:58.172752 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacd5c2a-b211-463e-bc39-799da9b88d1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:58.174375 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacd5c2a-b211-463e-bc39-799da9b88d1a admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:58.183906 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1187/2388] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:50:58 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:50:58.188570 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806a8612-f188-47f4-9b89-34e7293162d1 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:58.198788 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806a8612-f188-47f4-9b89-34e7293162d1 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:58.204011 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b88aa6-c692-4dc5-a72a-005226176445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:58.204916 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b88aa6-c692-4dc5-a72a-005226176445 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:58.205248 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b88aa6-c692-4dc5-a72a-005226176445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:58.205529 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b88aa6-c692-4dc5-a72a-005226176445 admin admin] PATH_INFO: `/v3/projects/21306330a45642d6855703b602e674df/users/fbb1af30f007422eaadbb44c32cdcba5/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:58.207536 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b88aa6-c692-4dc5-a72a-005226176445 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21306330a45642d6855703b602e674df, user_id=fbb1af30f007422eaadbb44c32cdcba5, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:58.210248 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b88aa6-c692-4dc5-a72a-005226176445 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:58.220289 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1202/2389] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:50:58 2022] PUT /identity/v3/projects/21306330a45642d6855703b602e674df/users/fbb1af30f007422eaadbb44c32cdcba5/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:50:58.231476 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190be6c-5317-4909-9214-43b2ffc7f640 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:58.231771 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190be6c-5317-4909-9214-43b2ffc7f640 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:58.232054 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190be6c-5317-4909-9214-43b2ffc7f640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:58.259862 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-f190be6c-5317-4909-9214-43b2ffc7f640 None None] MFA Rules not processed for user `fbb1af30f007422eaadbb44c32cdcba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:50:58.261092 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-f190be6c-5317-4909-9214-43b2ffc7f640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:50:58.330476 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1188/2390] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:50:58 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:50:58.341115 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b88aa6-c692-4dc5-a72a-005226176445 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:58.342892 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-01b88aa6-c692-4dc5-a72a-005226176445 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:50:58.404418 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b88aa6-c692-4dc5-a72a-005226176445 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:58.408525 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917986f9-3735-4635-86a4-eccc9e3c1d2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:58.409187 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917986f9-3735-4635-86a4-eccc9e3c1d2f service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:58.409477 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917986f9-3735-4635-86a4-eccc9e3c1d2f service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:58.409755 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917986f9-3735-4635-86a4-eccc9e3c1d2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:58.418600 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917986f9-3735-4635-86a4-eccc9e3c1d2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:58.420041 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917986f9-3735-4635-86a4-eccc9e3c1d2f service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:58.429963 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1203/2391] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:50:58 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:50:58.915406 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190be6c-5317-4909-9214-43b2ffc7f640 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:50:58.916835 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-f190be6c-5317-4909-9214-43b2ffc7f640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:50:58.929540 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190be6c-5317-4909-9214-43b2ffc7f640 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:50:58.933133 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bfc189-93c4-4aa7-8d59-b797637171db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:50:58.933804 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bfc189-93c4-4aa7-8d59-b797637171db service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:50:58.934092 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bfc189-93c4-4aa7-8d59-b797637171db service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:50:58.934371 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bfc189-93c4-4aa7-8d59-b797637171db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:50:58.960305 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:50:58.999903 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bfc189-93c4-4aa7-8d59-b797637171db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:50:59.001376 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bfc189-93c4-4aa7-8d59-b797637171db service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:50:59.011128 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1189/2392] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:50:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:51:03.002229 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917986f9-3735-4635-86a4-eccc9e3c1d2f service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:03.010717 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917986f9-3735-4635-86a4-eccc9e3c1d2f service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:03.014291 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1948900d-c5ab-409f-9193-463602c63520 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:03.014919 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1948900d-c5ab-409f-9193-463602c63520 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:03.015204 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1948900d-c5ab-409f-9193-463602c63520 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:03.015478 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1948900d-c5ab-409f-9193-463602c63520 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:03.070635 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1948900d-c5ab-409f-9193-463602c63520 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:03.072079 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1948900d-c5ab-409f-9193-463602c63520 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:03.080678 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1204/2393] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:51:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:51:05.154635 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bfc189-93c4-4aa7-8d59-b797637171db service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:05.217339 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bfc189-93c4-4aa7-8d59-b797637171db service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:05.222449 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45180a17-7e61-4303-a698-0de8e7fb28bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:05.223320 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45180a17-7e61-4303-a698-0de8e7fb28bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:05.223638 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45180a17-7e61-4303-a698-0de8e7fb28bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:05.223914 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45180a17-7e61-4303-a698-0de8e7fb28bd admin admin] PATH_INFO: `/v3/users/3f0ec213977b4f41b7c3c6af26a3fc79` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:05.225107 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45180a17-7e61-4303-a698-0de8e7fb28bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f0ec213977b4f41b7c3c6af26a3fc79)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:05.227058 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45180a17-7e61-4303-a698-0de8e7fb28bd admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:05.272276 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-45180a17-7e61-4303-a698-0de8e7fb28bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f0ec213977b4f41b7c3c6af26a3fc79'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:05.272943 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-45180a17-7e61-4303-a698-0de8e7fb28bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f0ec213977b4f41b7c3c6af26a3fc79'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:05.280050 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-45180a17-7e61-4303-a698-0de8e7fb28bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f0ec213977b4f41b7c3c6af26a3fc79'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:05.292653 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-45180a17-7e61-4303-a698-0de8e7fb28bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f0ec213977b4f41b7c3c6af26a3fc79'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:05.293338 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-45180a17-7e61-4303-a698-0de8e7fb28bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f0ec213977b4f41b7c3c6af26a3fc79'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:05.305269 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1190/2394] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:51:05 2022] DELETE /identity/v3/users/3f0ec213977b4f41b7c3c6af26a3fc79 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:51:05.309824 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77516dbb-cdf2-453b-8613-dc5a8018f67c None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:05.310306 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77516dbb-cdf2-453b-8613-dc5a8018f67c None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:05.310426 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77516dbb-cdf2-453b-8613-dc5a8018f67c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:05.327554 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-77516dbb-cdf2-453b-8613-dc5a8018f67c None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:51:05.328825 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-77516dbb-cdf2-453b-8613-dc5a8018f67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:51:05.373663 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:51:05.389712 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1205/2395] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:51:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:51:05.400244 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45180a17-7e61-4303-a698-0de8e7fb28bd admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:05.401752 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-45180a17-7e61-4303-a698-0de8e7fb28bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:51:05.449056 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:51:05.461880 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45180a17-7e61-4303-a698-0de8e7fb28bd admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:05.465766 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25495f0d-9bd8-4fdd-a86e-0de1ebbaa0a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:05.466408 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25495f0d-9bd8-4fdd-a86e-0de1ebbaa0a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:05.466694 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25495f0d-9bd8-4fdd-a86e-0de1ebbaa0a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:05.466970 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25495f0d-9bd8-4fdd-a86e-0de1ebbaa0a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:05.476898 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25495f0d-9bd8-4fdd-a86e-0de1ebbaa0a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:05.478404 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25495f0d-9bd8-4fdd-a86e-0de1ebbaa0a2 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:05.487942 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1191/2396] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:51:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:51:05.762524 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77516dbb-cdf2-453b-8613-dc5a8018f67c None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:05.763992 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-77516dbb-cdf2-453b-8613-dc5a8018f67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:51:05.776207 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77516dbb-cdf2-453b-8613-dc5a8018f67c None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:05.781041 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cacd6c4-bb8a-4533-b264-f08a67ce365b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:05.781948 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacd6c4-bb8a-4533-b264-f08a67ce365b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:05.782280 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacd6c4-bb8a-4533-b264-f08a67ce365b admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:05.782559 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacd6c4-bb8a-4533-b264-f08a67ce365b admin admin] PATH_INFO: `/v3/projects/830095fbe2d64d22bf132086d33c7f36` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:05.783789 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cacd6c4-bb8a-4533-b264-f08a67ce365b admin admin] RBAC: Authorizing `identity:delete_project(project_id=830095fbe2d64d22bf132086d33c7f36)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:05.786021 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cacd6c4-bb8a-4533-b264-f08a67ce365b admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:05.833496 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-2cacd6c4-bb8a-4533-b264-f08a67ce365b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '830095fbe2d64d22bf132086d33c7f36', 'request_id': 'req-2cacd6c4-bb8a-4533-b264-f08a67ce365b'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:05.838950 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-2cacd6c4-bb8a-4533-b264-f08a67ce365b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '830095fbe2d64d22bf132086d33c7f36', 'request_id': 'req-2cacd6c4-bb8a-4533-b264-f08a67ce365b'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:05.845058 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-2cacd6c4-bb8a-4533-b264-f08a67ce365b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '830095fbe2d64d22bf132086d33c7f36', 'request_id': 'req-2cacd6c4-bb8a-4533-b264-f08a67ce365b'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:05.857818 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-2cacd6c4-bb8a-4533-b264-f08a67ce365b admin admin] The token cache is being invalidate because project 830095fbe2d64d22bf132086d33c7f36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:51:05.858116 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-2cacd6c4-bb8a-4533-b264-f08a67ce365b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:05.858748 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-2cacd6c4-bb8a-4533-b264-f08a67ce365b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:05.860823 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1206/2397] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:51:05 2022] DELETE /identity/v3/projects/830095fbe2d64d22bf132086d33c7f36 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:51:05.864783 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25495f0d-9bd8-4fdd-a86e-0de1ebbaa0a2 service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:05.874601 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25495f0d-9bd8-4fdd-a86e-0de1ebbaa0a2 service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:05.939223 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ed4113-0d4c-4a86-917c-42fd80e67723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:05.940109 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ed4113-0d4c-4a86-917c-42fd80e67723 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:05.940400 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ed4113-0d4c-4a86-917c-42fd80e67723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:05.940680 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ed4113-0d4c-4a86-917c-42fd80e67723 admin admin] PATH_INFO: `/v3/users/2031d822edc8407bb7ffbc0be2fb52ad` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:05.942186 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ed4113-0d4c-4a86-917c-42fd80e67723 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2031d822edc8407bb7ffbc0be2fb52ad)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:05.944204 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ed4113-0d4c-4a86-917c-42fd80e67723 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:05.991219 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-23ed4113-0d4c-4a86-917c-42fd80e67723 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2031d822edc8407bb7ffbc0be2fb52ad'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:05.991855 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-23ed4113-0d4c-4a86-917c-42fd80e67723 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2031d822edc8407bb7ffbc0be2fb52ad'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:06.002863 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-23ed4113-0d4c-4a86-917c-42fd80e67723 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2031d822edc8407bb7ffbc0be2fb52ad'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:06.016313 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-23ed4113-0d4c-4a86-917c-42fd80e67723 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2031d822edc8407bb7ffbc0be2fb52ad'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:06.017036 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-23ed4113-0d4c-4a86-917c-42fd80e67723 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2031d822edc8407bb7ffbc0be2fb52ad'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:06.029770 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1192/2398] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:51:05 2022] DELETE /identity/v3/users/2031d822edc8407bb7ffbc0be2fb52ad => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:51:06.332375 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cacd6c4-bb8a-4533-b264-f08a67ce365b admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:06.334065 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-2cacd6c4-bb8a-4533-b264-f08a67ce365b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:51:06.348398 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cacd6c4-bb8a-4533-b264-f08a67ce365b admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:06.412033 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99358c81-2950-4cd8-97ce-473c4df933e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:06.412822 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99358c81-2950-4cd8-97ce-473c4df933e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:06.413109 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99358c81-2950-4cd8-97ce-473c4df933e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:06.413418 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99358c81-2950-4cd8-97ce-473c4df933e9 admin admin] PATH_INFO: `/v3/projects/92d6e4ad3c6b40bea5059dd027bc8a0f` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:06.414615 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99358c81-2950-4cd8-97ce-473c4df933e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92d6e4ad3c6b40bea5059dd027bc8a0f)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:06.416541 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99358c81-2950-4cd8-97ce-473c4df933e9 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:06.464840 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-99358c81-2950-4cd8-97ce-473c4df933e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92d6e4ad3c6b40bea5059dd027bc8a0f', 'request_id': 'req-99358c81-2950-4cd8-97ce-473c4df933e9'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:06.470792 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-99358c81-2950-4cd8-97ce-473c4df933e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92d6e4ad3c6b40bea5059dd027bc8a0f', 'request_id': 'req-99358c81-2950-4cd8-97ce-473c4df933e9'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:06.476796 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-99358c81-2950-4cd8-97ce-473c4df933e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92d6e4ad3c6b40bea5059dd027bc8a0f', 'request_id': 'req-99358c81-2950-4cd8-97ce-473c4df933e9'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:06.489033 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-99358c81-2950-4cd8-97ce-473c4df933e9 admin admin] The token cache is being invalidate because project 92d6e4ad3c6b40bea5059dd027bc8a0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:51:06.489351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-99358c81-2950-4cd8-97ce-473c4df933e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:06.489982 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-99358c81-2950-4cd8-97ce-473c4df933e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:06.491663 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1207/2399] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:51:06 2022] DELETE /identity/v3/projects/92d6e4ad3c6b40bea5059dd027bc8a0f => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:51:06.496019 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ed4113-0d4c-4a86-917c-42fd80e67723 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:06.497838 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-23ed4113-0d4c-4a86-917c-42fd80e67723 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:51:06.511515 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ed4113-0d4c-4a86-917c-42fd80e67723 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:06.567246 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8231e386-1b75-410a-b756-b8cdfdaeb765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:06.568216 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231e386-1b75-410a-b756-b8cdfdaeb765 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:06.568552 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231e386-1b75-410a-b756-b8cdfdaeb765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:06.568877 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231e386-1b75-410a-b756-b8cdfdaeb765 admin admin] PATH_INFO: `/v3/users/fbb1af30f007422eaadbb44c32cdcba5` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:06.570271 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8231e386-1b75-410a-b756-b8cdfdaeb765 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbb1af30f007422eaadbb44c32cdcba5)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:06.572303 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8231e386-1b75-410a-b756-b8cdfdaeb765 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:06.578282 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:51:06.622582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8231e386-1b75-410a-b756-b8cdfdaeb765 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbb1af30f007422eaadbb44c32cdcba5'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:06.623292 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8231e386-1b75-410a-b756-b8cdfdaeb765 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbb1af30f007422eaadbb44c32cdcba5'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:06.630599 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8231e386-1b75-410a-b756-b8cdfdaeb765 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbb1af30f007422eaadbb44c32cdcba5'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:06.647745 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8231e386-1b75-410a-b756-b8cdfdaeb765 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbb1af30f007422eaadbb44c32cdcba5'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:06.648471 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8231e386-1b75-410a-b756-b8cdfdaeb765 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbb1af30f007422eaadbb44c32cdcba5'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:06.661744 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1193/2400] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:51:06 2022] DELETE /identity/v3/users/fbb1af30f007422eaadbb44c32cdcba5 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:51:06.937263 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99358c81-2950-4cd8-97ce-473c4df933e9 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:06.939252 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-99358c81-2950-4cd8-97ce-473c4df933e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:51:06.949408 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:51:06.953926 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99358c81-2950-4cd8-97ce-473c4df933e9 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:07.013581 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b582965-f501-43b8-a8c7-9e556896f4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:07.014447 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b582965-f501-43b8-a8c7-9e556896f4cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:07.014730 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b582965-f501-43b8-a8c7-9e556896f4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:07.015005 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b582965-f501-43b8-a8c7-9e556896f4cc admin admin] PATH_INFO: `/v3/projects/21306330a45642d6855703b602e674df` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:07.016184 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b582965-f501-43b8-a8c7-9e556896f4cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=21306330a45642d6855703b602e674df)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:07.019074 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b582965-f501-43b8-a8c7-9e556896f4cc admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:07.068885 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6b582965-f501-43b8-a8c7-9e556896f4cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21306330a45642d6855703b602e674df', 'request_id': 'req-6b582965-f501-43b8-a8c7-9e556896f4cc'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:07.074315 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6b582965-f501-43b8-a8c7-9e556896f4cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21306330a45642d6855703b602e674df', 'request_id': 'req-6b582965-f501-43b8-a8c7-9e556896f4cc'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:07.080882 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6b582965-f501-43b8-a8c7-9e556896f4cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21306330a45642d6855703b602e674df', 'request_id': 'req-6b582965-f501-43b8-a8c7-9e556896f4cc'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:07.092938 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6b582965-f501-43b8-a8c7-9e556896f4cc admin admin] The token cache is being invalidate because project 21306330a45642d6855703b602e674df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:51:07.093408 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6b582965-f501-43b8-a8c7-9e556896f4cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:07.094052 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6b582965-f501-43b8-a8c7-9e556896f4cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:07.095776 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1208/2401] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:51:06 2022] DELETE /identity/v3/projects/21306330a45642d6855703b602e674df => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:51:07.102592 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb57d1-ae91-4963-8fe2-828c9891b173 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:07.102899 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb57d1-ae91-4963-8fe2-828c9891b173 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:07.103240 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb57d1-ae91-4963-8fe2-828c9891b173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:07.121399 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-bbeb57d1-ae91-4963-8fe2-828c9891b173 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:51:07.122850 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-bbeb57d1-ae91-4963-8fe2-828c9891b173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:51:07.180980 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1194/2402] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:51:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:51:07.189868 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81a1d9-417b-4d77-8321-931cdcbc347d None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:07.190165 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81a1d9-417b-4d77-8321-931cdcbc347d None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:07.190450 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81a1d9-417b-4d77-8321-931cdcbc347d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:07.209003 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-ea81a1d9-417b-4d77-8321-931cdcbc347d None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:51:07.210244 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-ea81a1d9-417b-4d77-8321-931cdcbc347d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:51:07.218135 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1209/2403] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:51:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:51:07.222902 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeb57d1-ae91-4963-8fe2-828c9891b173 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:07.232616 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeb57d1-ae91-4963-8fe2-828c9891b173 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:07.237346 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed53da7-f4d1-4708-9b41-ad7a8fa74e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:07.238003 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed53da7-f4d1-4708-9b41-ad7a8fa74e35 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:07.238285 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed53da7-f4d1-4708-9b41-ad7a8fa74e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:07.238560 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed53da7-f4d1-4708-9b41-ad7a8fa74e35 admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:07.239284 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed53da7-f4d1-4708-9b41-ad7a8fa74e35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:51:07.239596 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed53da7-f4d1-4708-9b41-ad7a8fa74e35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:07.241210 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed53da7-f4d1-4708-9b41-ad7a8fa74e35 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:07.253260 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1195/2404] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:51:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:51:07.257441 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea81a1d9-417b-4d77-8321-931cdcbc347d None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:07.267109 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea81a1d9-417b-4d77-8321-931cdcbc347d None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:07.271783 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f01f30-4ced-4c6a-9efd-fbfaad832ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:07.272432 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f01f30-4ced-4c6a-9efd-fbfaad832ae0 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:07.272722 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f01f30-4ced-4c6a-9efd-fbfaad832ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:07.273001 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f01f30-4ced-4c6a-9efd-fbfaad832ae0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:07.273741 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f01f30-4ced-4c6a-9efd-fbfaad832ae0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:07.275664 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f01f30-4ced-4c6a-9efd-fbfaad832ae0 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:07.305431 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1210/2405] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:51:07 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:51:07.306754 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed53da7-f4d1-4708-9b41-ad7a8fa74e35 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:07.317082 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed53da7-f4d1-4708-9b41-ad7a8fa74e35 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:07.374753 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5ac6a2-417e-4584-94f7-a5bb4ddcedfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:07.375423 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ac6a2-417e-4584-94f7-a5bb4ddcedfc admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:07.375710 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ac6a2-417e-4584-94f7-a5bb4ddcedfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:07.376033 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ac6a2-417e-4584-94f7-a5bb4ddcedfc admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:07.376738 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5ac6a2-417e-4584-94f7-a5bb4ddcedfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:07.378686 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5ac6a2-417e-4584-94f7-a5bb4ddcedfc admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:07.397804 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1196/2406] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:51:07 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:51:07.402085 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f01f30-4ced-4c6a-9efd-fbfaad832ae0 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:07.412948 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f01f30-4ced-4c6a-9efd-fbfaad832ae0 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:07.417676 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad794f1-da19-4871-9465-9bac2e0e5b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:07.418327 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad794f1-da19-4871-9465-9bac2e0e5b03 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:07.418609 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad794f1-da19-4871-9465-9bac2e0e5b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:07.418886 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad794f1-da19-4871-9465-9bac2e0e5b03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:07.419617 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad794f1-da19-4871-9465-9bac2e0e5b03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:07.421244 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad794f1-da19-4871-9465-9bac2e0e5b03 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:07.431474 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1211/2407] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:51:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:51:07.435501 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5ac6a2-417e-4584-94f7-a5bb4ddcedfc admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:07.440716 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:51:07.447047 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5ac6a2-417e-4584-94f7-a5bb4ddcedfc admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:07.452844 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76362c1-53a9-4750-bf6e-fc192af4d1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:07.453549 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76362c1-53a9-4750-bf6e-fc192af4d1fb admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:07.453839 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76362c1-53a9-4750-bf6e-fc192af4d1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:07.454626 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76362c1-53a9-4750-bf6e-fc192af4d1fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:07.455362 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76362c1-53a9-4750-bf6e-fc192af4d1fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:07.456965 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76362c1-53a9-4750-bf6e-fc192af4d1fb admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:07.467916 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1197/2408] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:51:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:51:07.472592 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad794f1-da19-4871-9465-9bac2e0e5b03 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:07.482438 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad794f1-da19-4871-9465-9bac2e0e5b03 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:07.487513 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6380e226-3a3b-4255-bf31-69f0921eaec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:07.488385 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6380e226-3a3b-4255-bf31-69f0921eaec5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:07.488676 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6380e226-3a3b-4255-bf31-69f0921eaec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:07.488945 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6380e226-3a3b-4255-bf31-69f0921eaec5 admin admin] PATH_INFO: `/v3/projects/c897a5e559fc477a84fcffd4a636b540/users/654f467192d349188667d5f65b770725/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:07.503861 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6380e226-3a3b-4255-bf31-69f0921eaec5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c897a5e559fc477a84fcffd4a636b540, user_id=654f467192d349188667d5f65b770725, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:07.506606 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6380e226-3a3b-4255-bf31-69f0921eaec5 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:07.519978 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1212/2409] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:51:07 2022] PUT /identity/v3/projects/c897a5e559fc477a84fcffd4a636b540/users/654f467192d349188667d5f65b770725/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:51:07.531120 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136d545-7d49-4b5d-808d-d1439c48c9e4 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:07.531597 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136d545-7d49-4b5d-808d-d1439c48c9e4 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:07.531702 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136d545-7d49-4b5d-808d-d1439c48c9e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:07.557603 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-0136d545-7d49-4b5d-808d-d1439c48c9e4 None None] MFA Rules not processed for user `654f467192d349188667d5f65b770725`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:51:07.558810 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-0136d545-7d49-4b5d-808d-d1439c48c9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:51:07.623874 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1198/2410] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:51:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:51:07.628912 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6380e226-3a3b-4255-bf31-69f0921eaec5 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:07.667226 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:51:07.697707 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6380e226-3a3b-4255-bf31-69f0921eaec5 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:07.702294 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44fe280-8562-4c83-bca2-b22ab707852e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:07.702950 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fe280-8562-4c83-bca2-b22ab707852e admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:07.703234 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fe280-8562-4c83-bca2-b22ab707852e admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:07.703539 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fe280-8562-4c83-bca2-b22ab707852e admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:07.704877 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44fe280-8562-4c83-bca2-b22ab707852e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:07.706861 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44fe280-8562-4c83-bca2-b22ab707852e admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:07.731451 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1213/2411] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:51:07 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:51:07.735566 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136d545-7d49-4b5d-808d-d1439c48c9e4 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:07.746037 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136d545-7d49-4b5d-808d-d1439c48c9e4 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:07.801785 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa097b3-5640-48fa-86c8-97d6e6cbe822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:07.802453 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa097b3-5640-48fa-86c8-97d6e6cbe822 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:07.802739 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa097b3-5640-48fa-86c8-97d6e6cbe822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:07.803015 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa097b3-5640-48fa-86c8-97d6e6cbe822 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:07.803703 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa097b3-5640-48fa-86c8-97d6e6cbe822 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:07.805635 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa097b3-5640-48fa-86c8-97d6e6cbe822 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:07.825463 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1199/2412] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:51:07 2022] POST /identity/v3/users => generated 289 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:51:07.830113 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44fe280-8562-4c83-bca2-b22ab707852e admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:07.840360 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44fe280-8562-4c83-bca2-b22ab707852e admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:07.844985 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a65ee-7abc-4284-b773-67a788f0d26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:07.845669 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a65ee-7abc-4284-b773-67a788f0d26e admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:07.845955 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a65ee-7abc-4284-b773-67a788f0d26e admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:07.846232 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a65ee-7abc-4284-b773-67a788f0d26e admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:07.846948 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a65ee-7abc-4284-b773-67a788f0d26e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:07.848573 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a65ee-7abc-4284-b773-67a788f0d26e admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:07.859527 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1214/2413] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:51:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:51:07.863449 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa097b3-5640-48fa-86c8-97d6e6cbe822 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:07.873457 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa097b3-5640-48fa-86c8-97d6e6cbe822 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:07.878877 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392defcc-4415-4ab7-95c0-e94acbdc6797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:07.879647 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392defcc-4415-4ab7-95c0-e94acbdc6797 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:07.879852 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392defcc-4415-4ab7-95c0-e94acbdc6797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:07.880130 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392defcc-4415-4ab7-95c0-e94acbdc6797 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:07.880888 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392defcc-4415-4ab7-95c0-e94acbdc6797 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:07.882529 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392defcc-4415-4ab7-95c0-e94acbdc6797 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:07.898707 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1200/2414] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:51:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:51:07.902959 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a65ee-7abc-4284-b773-67a788f0d26e admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:07.913640 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a65ee-7abc-4284-b773-67a788f0d26e admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:07.918715 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2a08f7-b51f-438f-9211-e34492fbe11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:07.919630 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a08f7-b51f-438f-9211-e34492fbe11d admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:07.920560 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a08f7-b51f-438f-9211-e34492fbe11d admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:07.920849 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a08f7-b51f-438f-9211-e34492fbe11d admin admin] PATH_INFO: `/v3/projects/fb212e3a1fdd4c1faa4367a476cad219/users/eb9cfcc2267d4f4b8be9c69441759133/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:07.938405 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2a08f7-b51f-438f-9211-e34492fbe11d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb212e3a1fdd4c1faa4367a476cad219, user_id=eb9cfcc2267d4f4b8be9c69441759133, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:07.941968 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2a08f7-b51f-438f-9211-e34492fbe11d admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:07.958027 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1215/2415] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:51:07 2022] PUT /identity/v3/projects/fb212e3a1fdd4c1faa4367a476cad219/users/eb9cfcc2267d4f4b8be9c69441759133/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:51:07.968893 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a7bed-7458-4592-a0a4-9877299d388b None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:07.969223 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a7bed-7458-4592-a0a4-9877299d388b None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:07.969599 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a7bed-7458-4592-a0a4-9877299d388b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:08.002740 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-2e0a7bed-7458-4592-a0a4-9877299d388b None None] MFA Rules not processed for user `eb9cfcc2267d4f4b8be9c69441759133`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:51:08.003957 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-2e0a7bed-7458-4592-a0a4-9877299d388b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:51:08.051924 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:51:08.077320 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1201/2416] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:51:07 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:51:08.078795 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2a08f7-b51f-438f-9211-e34492fbe11d admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:08.143362 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2a08f7-b51f-438f-9211-e34492fbe11d admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:08.148112 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac95cc56-fc24-4a7c-bb1b-783eb2773edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:08.148785 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95cc56-fc24-4a7c-bb1b-783eb2773edd admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:08.149084 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95cc56-fc24-4a7c-bb1b-783eb2773edd admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:08.149399 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95cc56-fc24-4a7c-bb1b-783eb2773edd admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:08.150114 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac95cc56-fc24-4a7c-bb1b-783eb2773edd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:08.152061 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac95cc56-fc24-4a7c-bb1b-783eb2773edd admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:08.179160 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1216/2417] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:51:08 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:51:08.182967 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a7bed-7458-4592-a0a4-9877299d388b None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:08.194292 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a7bed-7458-4592-a0a4-9877299d388b None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:08.253130 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa756811-0be5-44ba-ad73-022297f73940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:08.253763 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa756811-0be5-44ba-ad73-022297f73940 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:08.254051 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa756811-0be5-44ba-ad73-022297f73940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:08.254333 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa756811-0be5-44ba-ad73-022297f73940 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:08.255079 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa756811-0be5-44ba-ad73-022297f73940 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:08.257054 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa756811-0be5-44ba-ad73-022297f73940 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:08.263426 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac95cc56-fc24-4a7c-bb1b-783eb2773edd admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:08.265187 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-ac95cc56-fc24-4a7c-bb1b-783eb2773edd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:51:08.279787 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1202/2418] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:51:08 2022] POST /identity/v3/users => generated 296 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:51:08.283878 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa756811-0be5-44ba-ad73-022297f73940 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:08.293510 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa756811-0be5-44ba-ad73-022297f73940 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:08.298892 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf4a9c-1daf-4795-b6fd-fcaf95ad1bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:08.299697 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf4a9c-1daf-4795-b6fd-fcaf95ad1bc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:08.299891 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf4a9c-1daf-4795-b6fd-fcaf95ad1bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:08.300203 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf4a9c-1daf-4795-b6fd-fcaf95ad1bc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:08.300925 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcf4a9c-1daf-4795-b6fd-fcaf95ad1bc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:08.302568 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcf4a9c-1daf-4795-b6fd-fcaf95ad1bc3 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:08.313430 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1203/2419] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:51:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:51:08.317467 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf4a9c-1daf-4795-b6fd-fcaf95ad1bc3 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:08.327636 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf4a9c-1daf-4795-b6fd-fcaf95ad1bc3 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:08.332340 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19070e0a-9564-464f-b61b-ce94a3df2b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:08.333246 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19070e0a-9564-464f-b61b-ce94a3df2b4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:08.333537 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19070e0a-9564-464f-b61b-ce94a3df2b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:08.333824 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19070e0a-9564-464f-b61b-ce94a3df2b4a admin admin] PATH_INFO: `/v3/projects/76d84a538ee1497bb7f15dce04e87c36/users/2a24f47c894e40348b6f3f4ee4eb24f9/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:08.339158 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac95cc56-fc24-4a7c-bb1b-783eb2773edd admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:08.344385 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf01804-646b-4f73-91fe-6ac8e347ebe6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e158e5c87f4954aeb297f42dabdc16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:08.345850 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf01804-646b-4f73-91fe-6ac8e347ebe6 service placement] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:08.346135 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf01804-646b-4f73-91fe-6ac8e347ebe6 service placement] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:08.346415 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf01804-646b-4f73-91fe-6ac8e347ebe6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:08.348473 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-adf01804-646b-4f73-91fe-6ac8e347ebe6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:51:08.349756 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19070e0a-9564-464f-b61b-ce94a3df2b4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=76d84a538ee1497bb7f15dce04e87c36, user_id=2a24f47c894e40348b6f3f4ee4eb24f9, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:08.352439 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19070e0a-9564-464f-b61b-ce94a3df2b4a admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:08.361138 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:51:08.363888 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf01804-646b-4f73-91fe-6ac8e347ebe6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:08.365980 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf01804-646b-4f73-91fe-6ac8e347ebe6 service placement] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:08.368413 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1204/2420] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:51:08 2022] PUT /identity/v3/projects/76d84a538ee1497bb7f15dce04e87c36/users/2a24f47c894e40348b6f3f4ee4eb24f9/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:51:08.372789 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19070e0a-9564-464f-b61b-ce94a3df2b4a admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:08.378980 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1217/2421] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:51:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:51:08.455513 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19070e0a-9564-464f-b61b-ce94a3df2b4a admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:08.460836 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6597795b-a9c4-41e6-aa05-707c33d5e94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:08.461669 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6597795b-a9c4-41e6-aa05-707c33d5e94d admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:08.462067 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6597795b-a9c4-41e6-aa05-707c33d5e94d admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:08.462429 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6597795b-a9c4-41e6-aa05-707c33d5e94d admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:08.463251 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6597795b-a9c4-41e6-aa05-707c33d5e94d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:08.464955 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6597795b-a9c4-41e6-aa05-707c33d5e94d admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:08.476666 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1205/2422] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:51:08 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:51:08.480695 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf01804-646b-4f73-91fe-6ac8e347ebe6 service placement] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:08.491770 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf01804-646b-4f73-91fe-6ac8e347ebe6 service placement] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:08.497243 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a41d46-1b26-4328-9f8e-bb3369bd26e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:08.498146 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a41d46-1b26-4328-9f8e-bb3369bd26e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:08.498449 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a41d46-1b26-4328-9f8e-bb3369bd26e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:08.498717 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a41d46-1b26-4328-9f8e-bb3369bd26e7 admin admin] PATH_INFO: `/v3/projects/76d84a538ee1497bb7f15dce04e87c36/users/2a24f47c894e40348b6f3f4ee4eb24f9/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:08.500740 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a41d46-1b26-4328-9f8e-bb3369bd26e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76d84a538ee1497bb7f15dce04e87c36, user_id=2a24f47c894e40348b6f3f4ee4eb24f9, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:08.503522 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a41d46-1b26-4328-9f8e-bb3369bd26e7 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:08.513999 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1218/2423] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:51:08 2022] PUT /identity/v3/projects/76d84a538ee1497bb7f15dce04e87c36/users/2a24f47c894e40348b6f3f4ee4eb24f9/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:51:08.526213 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22dcbd0-8831-4d43-8d0f-f1ccd296ef4a None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:08.526585 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22dcbd0-8831-4d43-8d0f-f1ccd296ef4a None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:08.526773 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22dcbd0-8831-4d43-8d0f-f1ccd296ef4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:08.553863 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-c22dcbd0-8831-4d43-8d0f-f1ccd296ef4a None None] MFA Rules not processed for user `2a24f47c894e40348b6f3f4ee4eb24f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:51:08.555061 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-c22dcbd0-8831-4d43-8d0f-f1ccd296ef4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:51:08.583383 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:51:08.622968 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1206/2424] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:51:08 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:51:08.634393 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a41d46-1b26-4328-9f8e-bb3369bd26e7 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:08.636127 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-07a41d46-1b26-4328-9f8e-bb3369bd26e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:51:08.696642 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a41d46-1b26-4328-9f8e-bb3369bd26e7 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:08.700412 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4720bb9-eae2-4abe-a678-497119dc4253 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:08.701062 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4720bb9-eae2-4abe-a678-497119dc4253 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:08.701380 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4720bb9-eae2-4abe-a678-497119dc4253 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:08.701662 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4720bb9-eae2-4abe-a678-497119dc4253 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:08.717840 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4720bb9-eae2-4abe-a678-497119dc4253 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:08.720391 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4720bb9-eae2-4abe-a678-497119dc4253 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:08.732706 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1219/2425] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:51:08 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:51:08.777822 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22dcbd0-8831-4d43-8d0f-f1ccd296ef4a None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:08.779232 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-c22dcbd0-8831-4d43-8d0f-f1ccd296ef4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:51:08.791692 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22dcbd0-8831-4d43-8d0f-f1ccd296ef4a None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:08.795326 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab133cac-4221-4e72-9603-f0ab8f6c8c2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:08.796021 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab133cac-4221-4e72-9603-f0ab8f6c8c2d service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:08.796260 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab133cac-4221-4e72-9603-f0ab8f6c8c2d service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:08.796540 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab133cac-4221-4e72-9603-f0ab8f6c8c2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:08.853423 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab133cac-4221-4e72-9603-f0ab8f6c8c2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:08.854866 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab133cac-4221-4e72-9603-f0ab8f6c8c2d service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:08.863689 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1207/2426] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:51:08 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:51:11.730236 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4720bb9-eae2-4abe-a678-497119dc4253 service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:11.795818 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4720bb9-eae2-4abe-a678-497119dc4253 service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:11.800807 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98ae7e0-51e6-451e-bcf4-cc54f6e89d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:11.801707 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ae7e0-51e6-451e-bcf4-cc54f6e89d75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:11.801995 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ae7e0-51e6-451e-bcf4-cc54f6e89d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:11.802276 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ae7e0-51e6-451e-bcf4-cc54f6e89d75 admin admin] PATH_INFO: `/v3/users/654f467192d349188667d5f65b770725` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:11.804131 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98ae7e0-51e6-451e-bcf4-cc54f6e89d75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=654f467192d349188667d5f65b770725)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:11.807797 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98ae7e0-51e6-451e-bcf4-cc54f6e89d75 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:11.855749 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-e98ae7e0-51e6-451e-bcf4-cc54f6e89d75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '654f467192d349188667d5f65b770725'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:11.856396 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-e98ae7e0-51e6-451e-bcf4-cc54f6e89d75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '654f467192d349188667d5f65b770725'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:11.856990 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-e98ae7e0-51e6-451e-bcf4-cc54f6e89d75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '654f467192d349188667d5f65b770725'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:11.864784 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-e98ae7e0-51e6-451e-bcf4-cc54f6e89d75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '654f467192d349188667d5f65b770725'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:11.877303 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-e98ae7e0-51e6-451e-bcf4-cc54f6e89d75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '654f467192d349188667d5f65b770725'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:11.889476 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1220/2427] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:51:11 2022] DELETE /identity/v3/users/654f467192d349188667d5f65b770725 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:51:11.894282 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e6abf-e999-4322-8e21-ffbebaea96ce None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:11.894718 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e6abf-e999-4322-8e21-ffbebaea96ce None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:11.894837 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e6abf-e999-4322-8e21-ffbebaea96ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:11.912514 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-d99e6abf-e999-4322-8e21-ffbebaea96ce None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:51:11.913749 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-d99e6abf-e999-4322-8e21-ffbebaea96ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:51:11.972164 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1208/2428] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:51:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:51:11.981854 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98ae7e0-51e6-451e-bcf4-cc54f6e89d75 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:11.983233 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e98ae7e0-51e6-451e-bcf4-cc54f6e89d75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:51:11.990244 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:51:12.042488 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98ae7e0-51e6-451e-bcf4-cc54f6e89d75 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:12.046156 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7cf9d-8a1f-445c-b3ff-b36797041223 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:12.046738 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7cf9d-8a1f-445c-b3ff-b36797041223 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:12.047022 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7cf9d-8a1f-445c-b3ff-b36797041223 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:12.047301 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7cf9d-8a1f-445c-b3ff-b36797041223 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:12.056679 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f7cf9d-8a1f-445c-b3ff-b36797041223 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:12.058170 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f7cf9d-8a1f-445c-b3ff-b36797041223 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:12.067348 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1221/2429] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:51:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:51:12.336501 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e6abf-e999-4322-8e21-ffbebaea96ce None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:12.338244 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-d99e6abf-e999-4322-8e21-ffbebaea96ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:51:12.350229 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e6abf-e999-4322-8e21-ffbebaea96ce None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:12.355399 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154111f8-f6ab-4c97-ab29-5cb6812a1c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:12.356278 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154111f8-f6ab-4c97-ab29-5cb6812a1c2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:12.356598 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154111f8-f6ab-4c97-ab29-5cb6812a1c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:12.356868 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154111f8-f6ab-4c97-ab29-5cb6812a1c2e admin admin] PATH_INFO: `/v3/projects/c897a5e559fc477a84fcffd4a636b540` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:12.358099 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154111f8-f6ab-4c97-ab29-5cb6812a1c2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c897a5e559fc477a84fcffd4a636b540)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:12.360081 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154111f8-f6ab-4c97-ab29-5cb6812a1c2e admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:12.411692 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-154111f8-f6ab-4c97-ab29-5cb6812a1c2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c897a5e559fc477a84fcffd4a636b540', 'request_id': 'req-154111f8-f6ab-4c97-ab29-5cb6812a1c2e'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:12.417447 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-154111f8-f6ab-4c97-ab29-5cb6812a1c2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c897a5e559fc477a84fcffd4a636b540', 'request_id': 'req-154111f8-f6ab-4c97-ab29-5cb6812a1c2e'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:12.418635 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:51:12.425943 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-154111f8-f6ab-4c97-ab29-5cb6812a1c2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c897a5e559fc477a84fcffd4a636b540', 'request_id': 'req-154111f8-f6ab-4c97-ab29-5cb6812a1c2e'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:12.441023 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-154111f8-f6ab-4c97-ab29-5cb6812a1c2e admin admin] The token cache is being invalidate because project c897a5e559fc477a84fcffd4a636b540 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:51:12.441355 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-154111f8-f6ab-4c97-ab29-5cb6812a1c2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:12.442003 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-154111f8-f6ab-4c97-ab29-5cb6812a1c2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:12.443556 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1209/2430] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:51:12 2022] DELETE /identity/v3/projects/c897a5e559fc477a84fcffd4a636b540 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:51:12.447578 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7cf9d-8a1f-445c-b3ff-b36797041223 service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:12.457243 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7cf9d-8a1f-445c-b3ff-b36797041223 service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:12.514089 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df471a-87f4-4b26-aa6f-895a6615543e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:12.514961 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df471a-87f4-4b26-aa6f-895a6615543e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:12.515251 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df471a-87f4-4b26-aa6f-895a6615543e admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:12.515532 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df471a-87f4-4b26-aa6f-895a6615543e admin admin] PATH_INFO: `/v3/users/eb9cfcc2267d4f4b8be9c69441759133` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:12.516716 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56df471a-87f4-4b26-aa6f-895a6615543e admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb9cfcc2267d4f4b8be9c69441759133)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:12.518697 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56df471a-87f4-4b26-aa6f-895a6615543e admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:12.565453 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-56df471a-87f4-4b26-aa6f-895a6615543e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb9cfcc2267d4f4b8be9c69441759133'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:12.566106 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-56df471a-87f4-4b26-aa6f-895a6615543e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb9cfcc2267d4f4b8be9c69441759133'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:12.566710 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-56df471a-87f4-4b26-aa6f-895a6615543e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb9cfcc2267d4f4b8be9c69441759133'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:12.573891 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-56df471a-87f4-4b26-aa6f-895a6615543e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb9cfcc2267d4f4b8be9c69441759133'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:12.586816 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-56df471a-87f4-4b26-aa6f-895a6615543e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb9cfcc2267d4f4b8be9c69441759133'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:12.599404 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1222/2431] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:51:12 2022] DELETE /identity/v3/users/eb9cfcc2267d4f4b8be9c69441759133 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:51:12.894911 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154111f8-f6ab-4c97-ab29-5cb6812a1c2e admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:12.896054 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-154111f8-f6ab-4c97-ab29-5cb6812a1c2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:51:12.913974 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154111f8-f6ab-4c97-ab29-5cb6812a1c2e admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:12.972442 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23173633-c21e-41fd-b037-c5abf5520da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:12.973336 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23173633-c21e-41fd-b037-c5abf5520da6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:12.973623 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23173633-c21e-41fd-b037-c5abf5520da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:12.973901 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23173633-c21e-41fd-b037-c5abf5520da6 admin admin] PATH_INFO: `/v3/projects/fb212e3a1fdd4c1faa4367a476cad219` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:12.975078 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23173633-c21e-41fd-b037-c5abf5520da6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb212e3a1fdd4c1faa4367a476cad219)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:12.976993 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23173633-c21e-41fd-b037-c5abf5520da6 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:13.023144 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-23173633-c21e-41fd-b037-c5abf5520da6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb212e3a1fdd4c1faa4367a476cad219', 'request_id': 'req-23173633-c21e-41fd-b037-c5abf5520da6'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:13.028769 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-23173633-c21e-41fd-b037-c5abf5520da6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb212e3a1fdd4c1faa4367a476cad219', 'request_id': 'req-23173633-c21e-41fd-b037-c5abf5520da6'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:13.035653 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-23173633-c21e-41fd-b037-c5abf5520da6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb212e3a1fdd4c1faa4367a476cad219', 'request_id': 'req-23173633-c21e-41fd-b037-c5abf5520da6'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:13.052104 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-23173633-c21e-41fd-b037-c5abf5520da6 admin admin] The token cache is being invalidate because project fb212e3a1fdd4c1faa4367a476cad219 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:51:13.052397 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-23173633-c21e-41fd-b037-c5abf5520da6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:13.053059 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-23173633-c21e-41fd-b037-c5abf5520da6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:13.054653 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1210/2432] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:51:12 2022] DELETE /identity/v3/projects/fb212e3a1fdd4c1faa4367a476cad219 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:51:13.059561 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df471a-87f4-4b26-aa6f-895a6615543e admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:13.061139 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-56df471a-87f4-4b26-aa6f-895a6615543e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:51:13.073751 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df471a-87f4-4b26-aa6f-895a6615543e admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:13.099658 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:51:13.136391 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f44f54-8f3d-465b-b909-b4284378229d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:13.137326 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f44f54-8f3d-465b-b909-b4284378229d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:13.137624 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f44f54-8f3d-465b-b909-b4284378229d admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:13.138027 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f44f54-8f3d-465b-b909-b4284378229d admin admin] PATH_INFO: `/v3/users/2a24f47c894e40348b6f3f4ee4eb24f9` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:13.139285 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f44f54-8f3d-465b-b909-b4284378229d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a24f47c894e40348b6f3f4ee4eb24f9)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:13.141286 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f44f54-8f3d-465b-b909-b4284378229d admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:13.191368 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-44f44f54-8f3d-465b-b909-b4284378229d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a24f47c894e40348b6f3f4ee4eb24f9'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:13.192021 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-44f44f54-8f3d-465b-b909-b4284378229d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a24f47c894e40348b6f3f4ee4eb24f9'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:13.192938 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-44f44f54-8f3d-465b-b909-b4284378229d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a24f47c894e40348b6f3f4ee4eb24f9'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:13.201063 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-44f44f54-8f3d-465b-b909-b4284378229d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a24f47c894e40348b6f3f4ee4eb24f9'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:13.216207 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-44f44f54-8f3d-465b-b909-b4284378229d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a24f47c894e40348b6f3f4ee4eb24f9'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:13.229333 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1223/2433] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:51:13 2022] DELETE /identity/v3/users/2a24f47c894e40348b6f3f4ee4eb24f9 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:51:13.498349 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23173633-c21e-41fd-b037-c5abf5520da6 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:13.499816 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-23173633-c21e-41fd-b037-c5abf5520da6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:51:13.513079 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23173633-c21e-41fd-b037-c5abf5520da6 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:13.571134 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dbcf97-8bd2-4e5c-868b-ac7ee5652a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:13.571996 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbcf97-8bd2-4e5c-868b-ac7ee5652a89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:13.572277 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbcf97-8bd2-4e5c-868b-ac7ee5652a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:13.572553 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbcf97-8bd2-4e5c-868b-ac7ee5652a89 admin admin] PATH_INFO: `/v3/projects/76d84a538ee1497bb7f15dce04e87c36` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:13.573764 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dbcf97-8bd2-4e5c-868b-ac7ee5652a89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76d84a538ee1497bb7f15dce04e87c36)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:13.575658 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dbcf97-8bd2-4e5c-868b-ac7ee5652a89 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:13.597066 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:51:13.623137 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-47dbcf97-8bd2-4e5c-868b-ac7ee5652a89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76d84a538ee1497bb7f15dce04e87c36', 'request_id': 'req-47dbcf97-8bd2-4e5c-868b-ac7ee5652a89'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:13.629063 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-47dbcf97-8bd2-4e5c-868b-ac7ee5652a89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76d84a538ee1497bb7f15dce04e87c36', 'request_id': 'req-47dbcf97-8bd2-4e5c-868b-ac7ee5652a89'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:13.635482 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-47dbcf97-8bd2-4e5c-868b-ac7ee5652a89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76d84a538ee1497bb7f15dce04e87c36', 'request_id': 'req-47dbcf97-8bd2-4e5c-868b-ac7ee5652a89'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:13.649308 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-47dbcf97-8bd2-4e5c-868b-ac7ee5652a89 admin admin] The token cache is being invalidate because project 76d84a538ee1497bb7f15dce04e87c36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:51:13.649605 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-47dbcf97-8bd2-4e5c-868b-ac7ee5652a89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:13.650239 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-47dbcf97-8bd2-4e5c-868b-ac7ee5652a89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:13.651790 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1211/2434] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:51:13 2022] DELETE /identity/v3/projects/76d84a538ee1497bb7f15dce04e87c36 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:51:13.658242 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef36b88-1119-4e56-97de-01807d9c2fff None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:13.658582 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef36b88-1119-4e56-97de-01807d9c2fff None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:13.658875 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef36b88-1119-4e56-97de-01807d9c2fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:13.678843 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-1ef36b88-1119-4e56-97de-01807d9c2fff None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:51:13.680053 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-1ef36b88-1119-4e56-97de-01807d9c2fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:51:13.743932 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1224/2435] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:51:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:51:13.752528 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3048b4c-73e0-4190-8058-1bc8964bec5c None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:13.752824 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3048b4c-73e0-4190-8058-1bc8964bec5c None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:13.753107 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3048b4c-73e0-4190-8058-1bc8964bec5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:13.772547 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-f3048b4c-73e0-4190-8058-1bc8964bec5c None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:51:13.774217 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-f3048b4c-73e0-4190-8058-1bc8964bec5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:51:13.784244 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1212/2436] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:51:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:51:13.788654 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef36b88-1119-4e56-97de-01807d9c2fff None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:13.799317 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef36b88-1119-4e56-97de-01807d9c2fff None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:13.804162 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f1442e-713d-4f8e-acca-5a5eb8eafe85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:13.804829 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f1442e-713d-4f8e-acca-5a5eb8eafe85 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:13.805114 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f1442e-713d-4f8e-acca-5a5eb8eafe85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:13.805425 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f1442e-713d-4f8e-acca-5a5eb8eafe85 admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:13.806158 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f1442e-713d-4f8e-acca-5a5eb8eafe85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:51:13.806616 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f1442e-713d-4f8e-acca-5a5eb8eafe85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:13.808234 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f1442e-713d-4f8e-acca-5a5eb8eafe85 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:13.821195 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1225/2437] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:51:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:51:13.825230 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3048b4c-73e0-4190-8058-1bc8964bec5c None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:13.834744 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3048b4c-73e0-4190-8058-1bc8964bec5c None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:13.839611 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f3571-52b8-4161-b620-7354a4aa0d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:13.840293 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f3571-52b8-4161-b620-7354a4aa0d75 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:13.840574 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f3571-52b8-4161-b620-7354a4aa0d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:13.840852 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f3571-52b8-4161-b620-7354a4aa0d75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:13.841588 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585f3571-52b8-4161-b620-7354a4aa0d75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:13.843491 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585f3571-52b8-4161-b620-7354a4aa0d75 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:13.877026 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1213/2438] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:51:13 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:51:13.884075 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f1442e-713d-4f8e-acca-5a5eb8eafe85 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:13.892284 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f1442e-713d-4f8e-acca-5a5eb8eafe85 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:13.928144 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:51:13.960105 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603f2eb2-8af5-4489-affc-99236697fa69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:13.960831 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f2eb2-8af5-4489-affc-99236697fa69 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:13.961120 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f2eb2-8af5-4489-affc-99236697fa69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:13.961553 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f2eb2-8af5-4489-affc-99236697fa69 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:13.962268 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603f2eb2-8af5-4489-affc-99236697fa69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:13.964286 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603f2eb2-8af5-4489-affc-99236697fa69 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:13.991906 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1226/2439] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:51:13 2022] POST /identity/v3/users => generated 288 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:51:13.993082 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f3571-52b8-4161-b620-7354a4aa0d75 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:14.003764 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f3571-52b8-4161-b620-7354a4aa0d75 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:14.008617 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f00501f-8be8-40d1-b13b-a88782b76c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:14.009314 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00501f-8be8-40d1-b13b-a88782b76c4a admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:14.009602 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00501f-8be8-40d1-b13b-a88782b76c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:14.009883 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00501f-8be8-40d1-b13b-a88782b76c4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:14.010615 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f00501f-8be8-40d1-b13b-a88782b76c4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:14.012214 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f00501f-8be8-40d1-b13b-a88782b76c4a admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:14.023327 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1214/2440] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:51:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:51:14.027602 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603f2eb2-8af5-4489-affc-99236697fa69 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:14.038482 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603f2eb2-8af5-4489-affc-99236697fa69 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:14.043564 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b15678-5bc8-4a56-9665-5c9fe3533c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:14.044298 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b15678-5bc8-4a56-9665-5c9fe3533c7b admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:14.044685 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b15678-5bc8-4a56-9665-5c9fe3533c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:14.045028 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b15678-5bc8-4a56-9665-5c9fe3533c7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:14.045807 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b15678-5bc8-4a56-9665-5c9fe3533c7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:14.047431 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b15678-5bc8-4a56-9665-5c9fe3533c7b admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:14.057789 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1227/2441] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:51:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:51:14.062484 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f00501f-8be8-40d1-b13b-a88782b76c4a admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:14.076837 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f00501f-8be8-40d1-b13b-a88782b76c4a admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:14.084734 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf3c92c-a97e-46d7-bb5b-9de0d4744e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:14.085677 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf3c92c-a97e-46d7-bb5b-9de0d4744e39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:14.085962 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf3c92c-a97e-46d7-bb5b-9de0d4744e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:14.086240 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf3c92c-a97e-46d7-bb5b-9de0d4744e39 admin admin] PATH_INFO: `/v3/projects/399b9cd7242a4bcca15210149ca0dafc/users/53633c92c7384cb08724ba97d2d51a7f/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:14.108576 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf3c92c-a97e-46d7-bb5b-9de0d4744e39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=399b9cd7242a4bcca15210149ca0dafc, user_id=53633c92c7384cb08724ba97d2d51a7f, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:14.111508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf3c92c-a97e-46d7-bb5b-9de0d4744e39 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:14.129807 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1215/2442] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:51:14 2022] PUT /identity/v3/projects/399b9cd7242a4bcca15210149ca0dafc/users/53633c92c7384cb08724ba97d2d51a7f/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:51:14.195263 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04c2eb-bf4c-4b16-b9fc-d4790399b995 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:14.195711 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04c2eb-bf4c-4b16-b9fc-d4790399b995 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:14.195832 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04c2eb-bf4c-4b16-b9fc-d4790399b995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:14.224381 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-cb04c2eb-bf4c-4b16-b9fc-d4790399b995 None None] MFA Rules not processed for user `53633c92c7384cb08724ba97d2d51a7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:51:14.225629 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-cb04c2eb-bf4c-4b16-b9fc-d4790399b995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:51:14.295294 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1228/2443] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:51:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:51:14.307156 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf3c92c-a97e-46d7-bb5b-9de0d4744e39 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:14.308848 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-0cf3c92c-a97e-46d7-bb5b-9de0d4744e39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:51:14.365195 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf3c92c-a97e-46d7-bb5b-9de0d4744e39 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:14.368864 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c88e757-7f95-471a-8b9a-deda1afae756 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:14.369556 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c88e757-7f95-471a-8b9a-deda1afae756 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:14.369847 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c88e757-7f95-471a-8b9a-deda1afae756 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:14.370151 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c88e757-7f95-471a-8b9a-deda1afae756 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:14.379480 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c88e757-7f95-471a-8b9a-deda1afae756 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:14.380984 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c88e757-7f95-471a-8b9a-deda1afae756 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:14.389778 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1216/2444] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:51:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:51:24.291345 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=51,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:51:24.385790 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=69,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:51:31.938391 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d5805-0acd-4f37-b18b-210360e6d096 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:31.938925 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d5805-0acd-4f37-b18b-210360e6d096 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:31.938925 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d5805-0acd-4f37-b18b-210360e6d096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:31.956708 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-352d5805-0acd-4f37-b18b-210360e6d096 None None] MFA Rules not processed for user `4077d35bf1044be79bfaa013e13c530e`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:51:31.957936 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-352d5805-0acd-4f37-b18b-210360e6d096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:51:32.011645 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1229/2445] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:51:31 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:51:38.193663 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c88e757-7f95-471a-8b9a-deda1afae756 service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:38.255525 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c88e757-7f95-471a-8b9a-deda1afae756 service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:38.260148 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413b423e-74c5-4720-89db-f4494cde6f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:38.261015 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413b423e-74c5-4720-89db-f4494cde6f4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:38.261328 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413b423e-74c5-4720-89db-f4494cde6f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:38.261607 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413b423e-74c5-4720-89db-f4494cde6f4e admin admin] PATH_INFO: `/v3/users/53633c92c7384cb08724ba97d2d51a7f` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:38.262790 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413b423e-74c5-4720-89db-f4494cde6f4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=53633c92c7384cb08724ba97d2d51a7f)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:38.264756 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413b423e-74c5-4720-89db-f4494cde6f4e admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:38.310927 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-413b423e-74c5-4720-89db-f4494cde6f4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53633c92c7384cb08724ba97d2d51a7f'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:38.311580 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-413b423e-74c5-4720-89db-f4494cde6f4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53633c92c7384cb08724ba97d2d51a7f'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:38.318732 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-413b423e-74c5-4720-89db-f4494cde6f4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53633c92c7384cb08724ba97d2d51a7f'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:38.330449 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-413b423e-74c5-4720-89db-f4494cde6f4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53633c92c7384cb08724ba97d2d51a7f'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:38.331083 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-413b423e-74c5-4720-89db-f4494cde6f4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53633c92c7384cb08724ba97d2d51a7f'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:38.343175 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1217/2446] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:51:38 2022] DELETE /identity/v3/users/53633c92c7384cb08724ba97d2d51a7f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:51:38.348252 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d10eb-0292-4a8a-b01c-948a8ad15af1 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:38.348749 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d10eb-0292-4a8a-b01c-948a8ad15af1 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:38.348856 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d10eb-0292-4a8a-b01c-948a8ad15af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:38.366935 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-109d10eb-0292-4a8a-b01c-948a8ad15af1 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:51:38.368160 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-109d10eb-0292-4a8a-b01c-948a8ad15af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:51:38.430302 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1230/2447] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:51:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:51:38.439938 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413b423e-74c5-4720-89db-f4494cde6f4e admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:38.441443 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-413b423e-74c5-4720-89db-f4494cde6f4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:51:38.500707 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413b423e-74c5-4720-89db-f4494cde6f4e admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:38.504705 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f8f3ca-12f8-44a6-8f2a-ac1eb1c17ee0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:38.505399 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f8f3ca-12f8-44a6-8f2a-ac1eb1c17ee0 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:38.505692 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f8f3ca-12f8-44a6-8f2a-ac1eb1c17ee0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:38.505975 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f8f3ca-12f8-44a6-8f2a-ac1eb1c17ee0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:38.515570 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f8f3ca-12f8-44a6-8f2a-ac1eb1c17ee0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:38.517113 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f8f3ca-12f8-44a6-8f2a-ac1eb1c17ee0 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:38.527111 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1218/2448] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:51:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:51:38.806469 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d10eb-0292-4a8a-b01c-948a8ad15af1 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:38.808224 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-109d10eb-0292-4a8a-b01c-948a8ad15af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:51:38.826524 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d10eb-0292-4a8a-b01c-948a8ad15af1 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:38.832059 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7856253d-1178-4ab5-a25a-83c63bf9bdd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:38.833044 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7856253d-1178-4ab5-a25a-83c63bf9bdd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:38.833388 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7856253d-1178-4ab5-a25a-83c63bf9bdd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:38.833549 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7856253d-1178-4ab5-a25a-83c63bf9bdd8 admin admin] PATH_INFO: `/v3/projects/399b9cd7242a4bcca15210149ca0dafc` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:38.834931 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7856253d-1178-4ab5-a25a-83c63bf9bdd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=399b9cd7242a4bcca15210149ca0dafc)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:38.837102 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7856253d-1178-4ab5-a25a-83c63bf9bdd8 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:38.892533 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-7856253d-1178-4ab5-a25a-83c63bf9bdd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '399b9cd7242a4bcca15210149ca0dafc', 'request_id': 'req-7856253d-1178-4ab5-a25a-83c63bf9bdd8'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:38.899623 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-7856253d-1178-4ab5-a25a-83c63bf9bdd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '399b9cd7242a4bcca15210149ca0dafc', 'request_id': 'req-7856253d-1178-4ab5-a25a-83c63bf9bdd8'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:38.906279 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-7856253d-1178-4ab5-a25a-83c63bf9bdd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '399b9cd7242a4bcca15210149ca0dafc', 'request_id': 'req-7856253d-1178-4ab5-a25a-83c63bf9bdd8'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:38.923094 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-7856253d-1178-4ab5-a25a-83c63bf9bdd8 admin admin] The token cache is being invalidate because project 399b9cd7242a4bcca15210149ca0dafc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:51:38.923428 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-7856253d-1178-4ab5-a25a-83c63bf9bdd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:38.924022 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-7856253d-1178-4ab5-a25a-83c63bf9bdd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:51:38.926398 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1231/2449] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:51:38 2022] DELETE /identity/v3/projects/399b9cd7242a4bcca15210149ca0dafc => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:51:38.931493 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd280b02-e0bf-4e2f-918f-062a7313f2fa None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:38.931819 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd280b02-e0bf-4e2f-918f-062a7313f2fa None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:38.932165 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd280b02-e0bf-4e2f-918f-062a7313f2fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:38.950525 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-cd280b02-e0bf-4e2f-918f-062a7313f2fa None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:51:38.951853 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-cd280b02-e0bf-4e2f-918f-062a7313f2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:51:38.969565 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:51:39.014787 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1219/2450] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:51:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:51:39.022595 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a08a4-9734-40b5-83fb-5b5191f268c6 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:39.022887 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a08a4-9734-40b5-83fb-5b5191f268c6 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:39.023195 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a08a4-9734-40b5-83fb-5b5191f268c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:39.041783 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-fb4a08a4-9734-40b5-83fb-5b5191f268c6 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:51:39.042986 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-fb4a08a4-9734-40b5-83fb-5b5191f268c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:51:39.051943 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1232/2451] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:51:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:51:39.056011 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd280b02-e0bf-4e2f-918f-062a7313f2fa None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:39.065819 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd280b02-e0bf-4e2f-918f-062a7313f2fa None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:39.070500 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf23717-d542-44ea-8f78-550fd8b17197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:39.071166 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf23717-d542-44ea-8f78-550fd8b17197 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:39.071463 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf23717-d542-44ea-8f78-550fd8b17197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:39.071744 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf23717-d542-44ea-8f78-550fd8b17197 admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:39.072488 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf23717-d542-44ea-8f78-550fd8b17197 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:51:39.072801 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf23717-d542-44ea-8f78-550fd8b17197 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:39.074456 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf23717-d542-44ea-8f78-550fd8b17197 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:39.086864 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1220/2452] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:51:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:51:39.090957 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a08a4-9734-40b5-83fb-5b5191f268c6 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:39.100807 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a08a4-9734-40b5-83fb-5b5191f268c6 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:39.106291 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7922e8fa-3bd9-44c1-8dca-9a138b7f2c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:39.106963 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922e8fa-3bd9-44c1-8dca-9a138b7f2c03 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:39.107247 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922e8fa-3bd9-44c1-8dca-9a138b7f2c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:39.107520 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922e8fa-3bd9-44c1-8dca-9a138b7f2c03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:39.108224 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7922e8fa-3bd9-44c1-8dca-9a138b7f2c03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:39.110244 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7922e8fa-3bd9-44c1-8dca-9a138b7f2c03 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:39.130597 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:51:39.138033 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1233/2453] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:51:39 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:51:39.142328 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf23717-d542-44ea-8f78-550fd8b17197 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:39.152633 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf23717-d542-44ea-8f78-550fd8b17197 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:39.209658 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e79f46-bea3-4233-93e2-43dc68cc1f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:39.210342 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e79f46-bea3-4233-93e2-43dc68cc1f5d admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:39.210628 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e79f46-bea3-4233-93e2-43dc68cc1f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:39.210907 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e79f46-bea3-4233-93e2-43dc68cc1f5d admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:39.211643 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e79f46-bea3-4233-93e2-43dc68cc1f5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:39.213607 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e79f46-bea3-4233-93e2-43dc68cc1f5d admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:39.234885 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1221/2454] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:51:39 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:51:39.239637 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7922e8fa-3bd9-44c1-8dca-9a138b7f2c03 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:39.250335 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7922e8fa-3bd9-44c1-8dca-9a138b7f2c03 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:39.255088 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1446232-d175-41c4-b96b-de3dede168de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:39.255732 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1446232-d175-41c4-b96b-de3dede168de admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:39.256021 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1446232-d175-41c4-b96b-de3dede168de admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:39.256301 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1446232-d175-41c4-b96b-de3dede168de admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:39.257062 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1446232-d175-41c4-b96b-de3dede168de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:39.258700 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1446232-d175-41c4-b96b-de3dede168de admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:39.269425 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1234/2455] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:51:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:51:39.274108 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e79f46-bea3-4233-93e2-43dc68cc1f5d admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:39.284651 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e79f46-bea3-4233-93e2-43dc68cc1f5d admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:39.289367 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c54c3-0508-4ba3-9b90-cd221fc0adf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:39.290030 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c54c3-0508-4ba3-9b90-cd221fc0adf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:39.290316 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c54c3-0508-4ba3-9b90-cd221fc0adf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:39.290592 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c54c3-0508-4ba3-9b90-cd221fc0adf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:39.291324 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292c54c3-0508-4ba3-9b90-cd221fc0adf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:39.292925 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292c54c3-0508-4ba3-9b90-cd221fc0adf8 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:39.309430 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1222/2456] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:51:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:51:39.310650 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1446232-d175-41c4-b96b-de3dede168de admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:39.320921 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1446232-d175-41c4-b96b-de3dede168de admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:39.325667 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aed4b5-defa-439f-a002-415afd8f2f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:39.326568 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aed4b5-defa-439f-a002-415afd8f2f5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:39.326853 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aed4b5-defa-439f-a002-415afd8f2f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:39.327134 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aed4b5-defa-439f-a002-415afd8f2f5b admin admin] PATH_INFO: `/v3/projects/f232c172df994d3d909d7999238b1935/users/708dbabdda134292a8259982db495e1f/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:39.343276 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aed4b5-defa-439f-a002-415afd8f2f5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f232c172df994d3d909d7999238b1935, user_id=708dbabdda134292a8259982db495e1f, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:39.346659 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aed4b5-defa-439f-a002-415afd8f2f5b admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:39.359759 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1235/2457] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:51:39 2022] PUT /identity/v3/projects/f232c172df994d3d909d7999238b1935/users/708dbabdda134292a8259982db495e1f/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:51:39.365842 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d839594-da7e-4fb7-9b02-84b7fd7a15cf None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:39.366147 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d839594-da7e-4fb7-9b02-84b7fd7a15cf None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:39.366445 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d839594-da7e-4fb7-9b02-84b7fd7a15cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:39.384386 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-4d839594-da7e-4fb7-9b02-84b7fd7a15cf None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:51:39.385644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-4d839594-da7e-4fb7-9b02-84b7fd7a15cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:51:39.452278 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1223/2458] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:51:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:51:39.462960 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aed4b5-defa-439f-a002-415afd8f2f5b admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:39.464602 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e8aed4b5-defa-439f-a002-415afd8f2f5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:51:39.521395 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aed4b5-defa-439f-a002-415afd8f2f5b admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:39.525076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d693a52-54d3-4ff4-b9cf-19b56d68ea06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:39.525873 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d693a52-54d3-4ff4-b9cf-19b56d68ea06 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:39.526164 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d693a52-54d3-4ff4-b9cf-19b56d68ea06 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:39.526448 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d693a52-54d3-4ff4-b9cf-19b56d68ea06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:39.536757 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d693a52-54d3-4ff4-b9cf-19b56d68ea06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:39.538254 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d693a52-54d3-4ff4-b9cf-19b56d68ea06 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:39.547948 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1236/2459] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:51:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:51:44.859046 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f421e0e-63f4-43a5-8c9f-7fb78901d9f3 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:44.860692 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f421e0e-63f4-43a5-8c9f-7fb78901d9f3 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:44.861063 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f421e0e-63f4-43a5-8c9f-7fb78901d9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:44.910758 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-1f421e0e-63f4-43a5-8c9f-7fb78901d9f3 None None] MFA Rules not processed for user `708dbabdda134292a8259982db495e1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:51:44.912143 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-1f421e0e-63f4-43a5-8c9f-7fb78901d9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:51:44.943865 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:51:45.032899 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1224/2460] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:51:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:51:45.038092 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d693a52-54d3-4ff4-b9cf-19b56d68ea06 service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:45.051077 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d693a52-54d3-4ff4-b9cf-19b56d68ea06 service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:45.057514 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f49773-354a-4e39-b755-e5c57b983795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:45.059326 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f49773-354a-4e39-b755-e5c57b983795 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:45.060561 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f49773-354a-4e39-b755-e5c57b983795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:45.061033 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f49773-354a-4e39-b755-e5c57b983795 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:45.062002 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f49773-354a-4e39-b755-e5c57b983795 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:45.065459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f49773-354a-4e39-b755-e5c57b983795 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:45.108339 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1237/2461] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:51:45 2022] POST /identity/v3/projects => generated 373 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:51:45.112789 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f421e0e-63f4-43a5-8c9f-7fb78901d9f3 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:45.131811 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f421e0e-63f4-43a5-8c9f-7fb78901d9f3 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:45.251674 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182c23db-a151-4ed8-87bd-aa9058688c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:45.253316 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c23db-a151-4ed8-87bd-aa9058688c33 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:45.253812 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c23db-a151-4ed8-87bd-aa9058688c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:45.254263 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c23db-a151-4ed8-87bd-aa9058688c33 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:45.255170 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182c23db-a151-4ed8-87bd-aa9058688c33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:45.257951 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182c23db-a151-4ed8-87bd-aa9058688c33 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:45.289707 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1225/2462] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:51:45 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:51:45.293647 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f49773-354a-4e39-b755-e5c57b983795 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:45.323078 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f49773-354a-4e39-b755-e5c57b983795 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:45.331463 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b5572-4937-4be7-883e-80ed8553394d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:45.332550 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b5572-4937-4be7-883e-80ed8553394d admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:45.332996 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b5572-4937-4be7-883e-80ed8553394d admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:45.333466 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b5572-4937-4be7-883e-80ed8553394d admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:45.334367 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103b5572-4937-4be7-883e-80ed8553394d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:45.336698 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103b5572-4937-4be7-883e-80ed8553394d admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:45.352188 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1238/2463] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:51:45.356563 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182c23db-a151-4ed8-87bd-aa9058688c33 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:45.370692 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182c23db-a151-4ed8-87bd-aa9058688c33 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:45.378716 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bae1c8-a25c-4220-b241-2a8d5d38fd0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:45.380249 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bae1c8-a25c-4220-b241-2a8d5d38fd0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:45.380249 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bae1c8-a25c-4220-b241-2a8d5d38fd0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:45.380249 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bae1c8-a25c-4220-b241-2a8d5d38fd0c admin admin] PATH_INFO: `/v3/projects/34dfa2f0b34644ce84102414e307276e/users/8e962e3b904248068c708d4d07c72a89/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:45.400385 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bae1c8-a25c-4220-b241-2a8d5d38fd0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=34dfa2f0b34644ce84102414e307276e, user_id=8e962e3b904248068c708d4d07c72a89, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:45.403654 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bae1c8-a25c-4220-b241-2a8d5d38fd0c admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:45.431862 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b5572-4937-4be7-883e-80ed8553394d admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:45.432821 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1226/2464] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:51:45 2022] PUT /identity/v3/projects/34dfa2f0b34644ce84102414e307276e/users/8e962e3b904248068c708d4d07c72a89/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:51:45.445674 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b5572-4937-4be7-883e-80ed8553394d admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:45.511781 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2153ecab-9273-44fa-b930-28a7e254d94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:45.512465 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153ecab-9273-44fa-b930-28a7e254d94a admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:45.512755 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153ecab-9273-44fa-b930-28a7e254d94a admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:45.513039 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153ecab-9273-44fa-b930-28a7e254d94a admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:45.513828 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2153ecab-9273-44fa-b930-28a7e254d94a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:45.515464 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2153ecab-9273-44fa-b930-28a7e254d94a admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:45.529729 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1239/2465] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:51:45.530960 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bae1c8-a25c-4220-b241-2a8d5d38fd0c admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:45.542173 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bae1c8-a25c-4220-b241-2a8d5d38fd0c admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:45.547041 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a7a3a-cfac-4114-a220-1410a04acf76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:45.548087 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a7a3a-cfac-4114-a220-1410a04acf76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:45.548478 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a7a3a-cfac-4114-a220-1410a04acf76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:45.548865 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a7a3a-cfac-4114-a220-1410a04acf76 admin admin] PATH_INFO: `/v3/projects/34dfa2f0b34644ce84102414e307276e/users/8e962e3b904248068c708d4d07c72a89/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:45.551595 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a7a3a-cfac-4114-a220-1410a04acf76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34dfa2f0b34644ce84102414e307276e, user_id=8e962e3b904248068c708d4d07c72a89, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:45.554912 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a7a3a-cfac-4114-a220-1410a04acf76 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:45.566137 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1227/2466] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:51:45 2022] PUT /identity/v3/projects/34dfa2f0b34644ce84102414e307276e/users/8e962e3b904248068c708d4d07c72a89/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:51:45.945477 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2153ecab-9273-44fa-b930-28a7e254d94a admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:45.947788 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-2153ecab-9273-44fa-b930-28a7e254d94a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:51:46.009068 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:51:46.067795 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2153ecab-9273-44fa-b930-28a7e254d94a admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:46.174831 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee469ff-a737-4a99-9676-878e9ab7345d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:46.175867 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee469ff-a737-4a99-9676-878e9ab7345d service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:46.176361 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee469ff-a737-4a99-9676-878e9ab7345d service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:46.177037 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee469ff-a737-4a99-9676-878e9ab7345d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:46.179528 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-7ee469ff-a737-4a99-9676-878e9ab7345d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:51:46.194532 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee469ff-a737-4a99-9676-878e9ab7345d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:46.196944 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee469ff-a737-4a99-9676-878e9ab7345d service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:46.213226 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1240/2467] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:51:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:51:51.858647 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e0138-8fc0-4d7c-a294-d840df196c12 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:51.859184 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e0138-8fc0-4d7c-a294-d840df196c12 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:51.859354 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e0138-8fc0-4d7c-a294-d840df196c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:51.914852 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-870e0138-8fc0-4d7c-a294-d840df196c12 None None] MFA Rules not processed for user `8e962e3b904248068c708d4d07c72a89`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:51:51.916531 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-870e0138-8fc0-4d7c-a294-d840df196c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:51:52.019077 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:51:52.033117 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1228/2468] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:51:51 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:51:52.046999 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee469ff-a737-4a99-9676-878e9ab7345d service nova] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:52.059229 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee469ff-a737-4a99-9676-878e9ab7345d service nova] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:52.065968 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47afe8d7-e75b-4def-bfbf-59cf30c01ad6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:52.066898 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47afe8d7-e75b-4def-bfbf-59cf30c01ad6 service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:52.067343 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47afe8d7-e75b-4def-bfbf-59cf30c01ad6 service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:52.067801 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47afe8d7-e75b-4def-bfbf-59cf30c01ad6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:52.154645 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47afe8d7-e75b-4def-bfbf-59cf30c01ad6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:52.156477 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47afe8d7-e75b-4def-bfbf-59cf30c01ad6 service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:52.169123 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1241/2469] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:51:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:51:53.735660 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c97040-eed2-405d-897d-52e47eb594bc None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:53.736115 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c97040-eed2-405d-897d-52e47eb594bc None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:53.736356 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c97040-eed2-405d-897d-52e47eb594bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:53.755272 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-34c97040-eed2-405d-897d-52e47eb594bc None None] MFA Rules not processed for user `708dbabdda134292a8259982db495e1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:51:53.756519 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-34c97040-eed2-405d-897d-52e47eb594bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:51:53.763593 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1229/2470] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:51:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:51:54.146109 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9dc9f6-b871-40a7-9ced-b6a3190a2c04 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:54.146109 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9dc9f6-b871-40a7-9ced-b6a3190a2c04 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:54.146711 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9dc9f6-b871-40a7-9ced-b6a3190a2c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:54.176341 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-0e9dc9f6-b871-40a7-9ced-b6a3190a2c04 None None] MFA Rules not processed for user `a30dc1dd2b00463bb3c7957864c56043`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:51:54.178367 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-0e9dc9f6-b871-40a7-9ced-b6a3190a2c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:51:54.245113 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:51:54.254651 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1242/2471] 10.222.0.36 () {58 vars in 967 bytes} [Thu May 19 16:51:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:51:54.304874 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c97040-eed2-405d-897d-52e47eb594bc None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:51:54.306657 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-34c97040-eed2-405d-897d-52e47eb594bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:51:54.422489 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c97040-eed2-405d-897d-52e47eb594bc None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:51:54.428924 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0439a3e-96ed-4eeb-a191-b881e991ccf5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffe7cebc6f3b40d8a8e1866ed2da9822', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:51:54.429839 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0439a3e-96ed-4eeb-a191-b881e991ccf5 service swift] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:51:54.430279 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0439a3e-96ed-4eeb-a191-b881e991ccf5 service swift] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:51:54.430707 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0439a3e-96ed-4eeb-a191-b881e991ccf5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:51:54.443986 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0439a3e-96ed-4eeb-a191-b881e991ccf5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:51:54.445927 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0439a3e-96ed-4eeb-a191-b881e991ccf5 service swift] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:51:54.457335 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1230/2472] 10.222.0.36 () {60 vars in 1384 bytes} [Thu May 19 16:51:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 16:52:02.345282 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9dc9f6-b871-40a7-9ced-b6a3190a2c04 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:52:02.347337 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-0e9dc9f6-b871-40a7-9ced-b6a3190a2c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:52:02.421414 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9dc9f6-b871-40a7-9ced-b6a3190a2c04 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:52:02.426298 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12336638-6ce5-4d88-b258-62da54b40042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:52:02.427210 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12336638-6ce5-4d88-b258-62da54b40042 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:52:02.427507 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12336638-6ce5-4d88-b258-62da54b40042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:52:02.427790 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12336638-6ce5-4d88-b258-62da54b40042 admin admin] PATH_INFO: `/v3/users/39a11fe9bc8549639e7e13b6f490641c` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:52:02.429071 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12336638-6ce5-4d88-b258-62da54b40042 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39a11fe9bc8549639e7e13b6f490641c)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:52:02.431076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12336638-6ce5-4d88-b258-62da54b40042 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:52:02.487358 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-12336638-6ce5-4d88-b258-62da54b40042 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39a11fe9bc8549639e7e13b6f490641c'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:52:02.488038 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-12336638-6ce5-4d88-b258-62da54b40042 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39a11fe9bc8549639e7e13b6f490641c'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:52:02.488657 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-12336638-6ce5-4d88-b258-62da54b40042 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39a11fe9bc8549639e7e13b6f490641c'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:52:02.496467 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-12336638-6ce5-4d88-b258-62da54b40042 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39a11fe9bc8549639e7e13b6f490641c'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:52:02.508670 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-12336638-6ce5-4d88-b258-62da54b40042 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39a11fe9bc8549639e7e13b6f490641c'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:52:02.522512 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1243/2473] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:52:02 2022] DELETE /identity/v3/users/39a11fe9bc8549639e7e13b6f490641c => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:52:02.803407 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0439a3e-96ed-4eeb-a191-b881e991ccf5 service swift] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:52:02.814050 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0439a3e-96ed-4eeb-a191-b881e991ccf5 service swift] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:52:02.899753 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5116cf35-4e3a-4ee5-a52b-acf93368c57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:52:02.900601 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116cf35-4e3a-4ee5-a52b-acf93368c57d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:52:02.900888 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116cf35-4e3a-4ee5-a52b-acf93368c57d admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:52:02.901174 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116cf35-4e3a-4ee5-a52b-acf93368c57d admin admin] PATH_INFO: `/v3/projects/66c1f23fbe114e9885266a0092cca13c` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:52:02.902394 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5116cf35-4e3a-4ee5-a52b-acf93368c57d admin admin] RBAC: Authorizing `identity:delete_project(project_id=66c1f23fbe114e9885266a0092cca13c)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:52:02.904611 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5116cf35-4e3a-4ee5-a52b-acf93368c57d admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:52:02.957319 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-5116cf35-4e3a-4ee5-a52b-acf93368c57d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66c1f23fbe114e9885266a0092cca13c', 'request_id': 'req-5116cf35-4e3a-4ee5-a52b-acf93368c57d'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:52:02.962796 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-5116cf35-4e3a-4ee5-a52b-acf93368c57d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66c1f23fbe114e9885266a0092cca13c', 'request_id': 'req-5116cf35-4e3a-4ee5-a52b-acf93368c57d'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:52:02.969780 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-5116cf35-4e3a-4ee5-a52b-acf93368c57d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66c1f23fbe114e9885266a0092cca13c', 'request_id': 'req-5116cf35-4e3a-4ee5-a52b-acf93368c57d'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:52:02.982473 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-5116cf35-4e3a-4ee5-a52b-acf93368c57d admin admin] The token cache is being invalidate because project 66c1f23fbe114e9885266a0092cca13c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:52:02.982760 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-5116cf35-4e3a-4ee5-a52b-acf93368c57d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:52:02.983401 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-5116cf35-4e3a-4ee5-a52b-acf93368c57d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:52:02.984951 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1231/2474] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:52:02 2022] DELETE /identity/v3/projects/66c1f23fbe114e9885266a0092cca13c => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:52:02.989544 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12336638-6ce5-4d88-b258-62da54b40042 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:52:02.990928 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-12336638-6ce5-4d88-b258-62da54b40042 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:52:03.005471 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12336638-6ce5-4d88-b258-62da54b40042 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:52:03.063570 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4730ac7-c47f-4f0e-9dec-76598a5f7aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:52:03.064451 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4730ac7-c47f-4f0e-9dec-76598a5f7aaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:52:03.064736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4730ac7-c47f-4f0e-9dec-76598a5f7aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:52:03.065014 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4730ac7-c47f-4f0e-9dec-76598a5f7aaa admin admin] PATH_INFO: `/v3/users/2af2f8e4af214c64a3681b7b6b3fd13c` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:52:03.066242 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4730ac7-c47f-4f0e-9dec-76598a5f7aaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=2af2f8e4af214c64a3681b7b6b3fd13c)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:52:03.068180 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4730ac7-c47f-4f0e-9dec-76598a5f7aaa admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:52:03.089943 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:52:03.115721 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-b4730ac7-c47f-4f0e-9dec-76598a5f7aaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2af2f8e4af214c64a3681b7b6b3fd13c'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:52:03.116382 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-b4730ac7-c47f-4f0e-9dec-76598a5f7aaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2af2f8e4af214c64a3681b7b6b3fd13c'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:52:03.117134 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-b4730ac7-c47f-4f0e-9dec-76598a5f7aaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2af2f8e4af214c64a3681b7b6b3fd13c'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:52:03.125635 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-b4730ac7-c47f-4f0e-9dec-76598a5f7aaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2af2f8e4af214c64a3681b7b6b3fd13c'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:52:03.137587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-b4730ac7-c47f-4f0e-9dec-76598a5f7aaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2af2f8e4af214c64a3681b7b6b3fd13c'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:52:03.150109 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1244/2475] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:52:02 2022] DELETE /identity/v3/users/2af2f8e4af214c64a3681b7b6b3fd13c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:52:03.325704 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5116cf35-4e3a-4ee5-a52b-acf93368c57d admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:52:03.327245 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-5116cf35-4e3a-4ee5-a52b-acf93368c57d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:52:03.341843 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5116cf35-4e3a-4ee5-a52b-acf93368c57d admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:52:03.347375 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:52:03.402813 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9553af1e-e629-4133-b26f-b4efca1e4282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:52:03.403594 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553af1e-e629-4133-b26f-b4efca1e4282 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:52:03.403882 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553af1e-e629-4133-b26f-b4efca1e4282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:52:03.404163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553af1e-e629-4133-b26f-b4efca1e4282 admin admin] PATH_INFO: `/v3/projects/6c47797a5c954e76a8a79974b42df0d0` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:52:03.405468 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9553af1e-e629-4133-b26f-b4efca1e4282 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c47797a5c954e76a8a79974b42df0d0)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:52:03.407377 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9553af1e-e629-4133-b26f-b4efca1e4282 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:52:03.456642 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-9553af1e-e629-4133-b26f-b4efca1e4282 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c47797a5c954e76a8a79974b42df0d0', 'request_id': 'req-9553af1e-e629-4133-b26f-b4efca1e4282'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:52:03.463740 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-9553af1e-e629-4133-b26f-b4efca1e4282 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c47797a5c954e76a8a79974b42df0d0', 'request_id': 'req-9553af1e-e629-4133-b26f-b4efca1e4282'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:52:03.471017 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-9553af1e-e629-4133-b26f-b4efca1e4282 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c47797a5c954e76a8a79974b42df0d0', 'request_id': 'req-9553af1e-e629-4133-b26f-b4efca1e4282'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:52:03.484350 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-9553af1e-e629-4133-b26f-b4efca1e4282 admin admin] The token cache is being invalidate because project 6c47797a5c954e76a8a79974b42df0d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:52:03.484891 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-9553af1e-e629-4133-b26f-b4efca1e4282 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:52:03.485464 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-9553af1e-e629-4133-b26f-b4efca1e4282 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:52:03.487375 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1232/2476] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:52:03 2022] DELETE /identity/v3/projects/6c47797a5c954e76a8a79974b42df0d0 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:52:13.146523 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:DELETE=8,keystone:SELECT=20,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:52:13.481000 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:52:15.670903 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb3cb62-0d52-461d-9ada-3cab37ec5349 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:52:15.671442 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb3cb62-0d52-461d-9ada-3cab37ec5349 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:52:15.671658 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb3cb62-0d52-461d-9ada-3cab37ec5349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:52:15.690459 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-4bb3cb62-0d52-461d-9ada-3cab37ec5349 None None] MFA Rules not processed for user `a30dc1dd2b00463bb3c7957864c56043`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:52:15.691743 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-4bb3cb62-0d52-461d-9ada-3cab37ec5349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:52:15.759425 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1245/2477] 10.222.0.36 () {58 vars in 1009 bytes} [Thu May 19 16:52:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:52:15.772589 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9553af1e-e629-4133-b26f-b4efca1e4282 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:52:15.774102 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-9553af1e-e629-4133-b26f-b4efca1e4282 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:52:15.848841 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9553af1e-e629-4133-b26f-b4efca1e4282 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:52:15.856163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4053139a-2618-4612-8a4d-1aeb5f56e449 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffe7cebc6f3b40d8a8e1866ed2da9822', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:52:15.856832 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053139a-2618-4612-8a4d-1aeb5f56e449 service swift] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:52:15.857121 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053139a-2618-4612-8a4d-1aeb5f56e449 service swift] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:52:15.857436 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053139a-2618-4612-8a4d-1aeb5f56e449 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:52:15.866263 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4053139a-2618-4612-8a4d-1aeb5f56e449 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:52:15.867895 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4053139a-2618-4612-8a4d-1aeb5f56e449 service swift] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:52:15.876167 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1233/2478] 10.222.0.36 () {60 vars in 1384 bytes} [Thu May 19 16:52:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 16:52:25.753873 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=25 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:52:25.872977 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=32 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:52:29.428296 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e5ad09-68f4-4f8e-b03a-a9c4e9484c05 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:52:29.428859 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e5ad09-68f4-4f8e-b03a-a9c4e9484c05 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:52:29.428859 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e5ad09-68f4-4f8e-b03a-a9c4e9484c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:52:29.448105 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-18e5ad09-68f4-4f8e-b03a-a9c4e9484c05 None None] MFA Rules not processed for user `0695c868a2d844089fcda29ea3653d5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:52:29.449351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-18e5ad09-68f4-4f8e-b03a-a9c4e9484c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:52:29.512137 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1246/2479] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:52:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:52:32.137216 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f434d4-2db2-4b8b-813b-47fb417dc8ac None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:52:32.137717 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f434d4-2db2-4b8b-813b-47fb417dc8ac None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:52:32.137823 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f434d4-2db2-4b8b-813b-47fb417dc8ac None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:52:32.139220 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1234/2480] 10.222.0.36 () {54 vars in 921 bytes} [Thu May 19 16:52:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:52:32.146515 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4991da-3c76-40bc-a8ef-450222406b81 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:52:32.147087 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4991da-3c76-40bc-a8ef-450222406b81 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:52:32.147356 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4991da-3c76-40bc-a8ef-450222406b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:52:32.169262 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-eb4991da-3c76-40bc-a8ef-450222406b81 None None] MFA Rules not processed for user `58bdeada00b74b7b8e186e6dc51e6dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:52:32.170769 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-eb4991da-3c76-40bc-a8ef-450222406b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:52:32.283588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1247/2481] 10.222.0.36 () {58 vars in 1018 bytes} [Thu May 19 16:52:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:52:32.301224 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f434d4-2db2-4b8b-813b-47fb417dc8ac None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:52:32.303571 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-f6f434d4-2db2-4b8b-813b-47fb417dc8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:52:32.374425 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f434d4-2db2-4b8b-813b-47fb417dc8ac None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:52:32.378045 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb17e4b3-c6fa-4904-ba91-b3a411e69b1c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57a5d7151478434f86be085bcc487f0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:52:32.378699 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17e4b3-c6fa-4904-ba91-b3a411e69b1c service glance] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:52:32.378986 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17e4b3-c6fa-4904-ba91-b3a411e69b1c service glance] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:52:32.379267 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17e4b3-c6fa-4904-ba91-b3a411e69b1c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:52:32.389787 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb17e4b3-c6fa-4904-ba91-b3a411e69b1c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:52:32.391247 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb17e4b3-c6fa-4904-ba91-b3a411e69b1c service glance] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:52:32.403047 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1235/2482] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:52:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:52:32.478309 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b845a-d3ee-49fb-bfe0-edf301ce7804 None None] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:52:32.478605 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b845a-d3ee-49fb-bfe0-edf301ce7804 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:52:32.478890 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b845a-d3ee-49fb-bfe0-edf301ce7804 None None] PATH_INFO: `/` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:52:32.480297 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1248/2483] 10.222.0.36 () {54 vars in 921 bytes} [Thu May 19 16:52:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 16:52:32.485604 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f4655-b019-409c-9e8d-e57831ae5668 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:52:32.485869 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f4655-b019-409c-9e8d-e57831ae5668 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:52:32.486176 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f4655-b019-409c-9e8d-e57831ae5668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:52:32.504709 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-8e3f4655-b019-409c-9e8d-e57831ae5668 None None] MFA Rules not processed for user `33472643b65e4fe08fd5b3aa2c618133`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:52:32.505957 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-8e3f4655-b019-409c-9e8d-e57831ae5668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:52:32.557607 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1236/2484] 10.222.0.36 () {58 vars in 1018 bytes} [Thu May 19 16:52:32 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:52:32.569790 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b845a-d3ee-49fb-bfe0-edf301ce7804 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:52:32.571480 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e71b845a-d3ee-49fb-bfe0-edf301ce7804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:52:32.585009 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b845a-d3ee-49fb-bfe0-edf301ce7804 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:52:32.588524 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f72fe44-346e-46b2-8cfb-70813450ea61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:52:32.589175 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f72fe44-346e-46b2-8cfb-70813450ea61 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:52:32.589475 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f72fe44-346e-46b2-8cfb-70813450ea61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:52:32.589757 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f72fe44-346e-46b2-8cfb-70813450ea61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:52:32.597403 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f72fe44-346e-46b2-8cfb-70813450ea61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:52:32.598845 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f72fe44-346e-46b2-8cfb-70813450ea61 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:52:32.606690 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1249/2485] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:52:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:52:32.690416 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa28fa42-7077-4e21-b17b-fc48bd2c62ec None None] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:52:32.690707 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa28fa42-7077-4e21-b17b-fc48bd2c62ec None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:52:32.690991 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa28fa42-7077-4e21-b17b-fc48bd2c62ec None None] PATH_INFO: `/` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:52:32.692413 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1237/2486] 10.222.0.36 () {54 vars in 921 bytes} [Thu May 19 16:52:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 16:52:32.697817 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed4c87-1193-4b06-9641-e5dfe35d4eee None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:52:32.698114 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed4c87-1193-4b06-9641-e5dfe35d4eee None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:52:32.698397 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed4c87-1193-4b06-9641-e5dfe35d4eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:52:32.717297 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-19ed4c87-1193-4b06-9641-e5dfe35d4eee None None] MFA Rules not processed for user `58bdeada00b74b7b8e186e6dc51e6dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:52:32.718513 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-19ed4c87-1193-4b06-9641-e5dfe35d4eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:52:32.728045 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1250/2487] 10.222.0.36 () {58 vars in 1018 bytes} [Thu May 19 16:52:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:52:32.738749 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa28fa42-7077-4e21-b17b-fc48bd2c62ec None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:52:32.749902 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa28fa42-7077-4e21-b17b-fc48bd2c62ec None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:52:32.754580 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d581d12-e0c7-482e-84f4-2d4d9af169bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:52:32.755344 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d581d12-e0c7-482e-84f4-2d4d9af169bd service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:52:32.755635 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d581d12-e0c7-482e-84f4-2d4d9af169bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:52:32.755918 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d581d12-e0c7-482e-84f4-2d4d9af169bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:52:32.767309 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d581d12-e0c7-482e-84f4-2d4d9af169bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:52:32.769713 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d581d12-e0c7-482e-84f4-2d4d9af169bd service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:52:32.781910 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1238/2488] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:52:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:52:34.462098 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487db6f-069c-4cb4-ad62-d74a80d2bee0 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:52:34.462629 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487db6f-069c-4cb4-ad62-d74a80d2bee0 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:52:34.462757 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487db6f-069c-4cb4-ad62-d74a80d2bee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:52:34.483192 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-e487db6f-069c-4cb4-ad62-d74a80d2bee0 None None] MFA Rules not processed for user `a30dc1dd2b00463bb3c7957864c56043`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:52:34.484481 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e487db6f-069c-4cb4-ad62-d74a80d2bee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:52:34.492591 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1251/2489] 10.222.0.36 () {58 vars in 1009 bytes} [Thu May 19 16:52:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:52:34.505760 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d581d12-e0c7-482e-84f4-2d4d9af169bd service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:52:34.521976 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d581d12-e0c7-482e-84f4-2d4d9af169bd service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:52:34.527535 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4d64e8-b463-44f0-844a-cbb329c0a858 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffe7cebc6f3b40d8a8e1866ed2da9822', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:52:34.528321 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d64e8-b463-44f0-844a-cbb329c0a858 service swift] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:52:34.528684 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d64e8-b463-44f0-844a-cbb329c0a858 service swift] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:52:34.529027 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d64e8-b463-44f0-844a-cbb329c0a858 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:52:34.539110 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4d64e8-b463-44f0-844a-cbb329c0a858 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:52:34.540842 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4d64e8-b463-44f0-844a-cbb329c0a858 service swift] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:52:34.551392 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1239/2490] 10.222.0.36 () {60 vars in 1384 bytes} [Thu May 19 16:52:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 16:52:44.477857 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=68 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:52:44.547464 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=61 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:52:53.286170 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e487db6f-069c-4cb4-ad62-d74a80d2bee0 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:52:53.287661 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e487db6f-069c-4cb4-ad62-d74a80d2bee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:52:53.298854 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e487db6f-069c-4cb4-ad62-d74a80d2bee0 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:52:53.302536 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68868b27-d397-428a-a7a2-e02e8dc9e40f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:52:53.303176 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68868b27-d397-428a-a7a2-e02e8dc9e40f service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:52:53.303460 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68868b27-d397-428a-a7a2-e02e8dc9e40f service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:52:53.303736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68868b27-d397-428a-a7a2-e02e8dc9e40f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:52:53.305494 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-68868b27-d397-428a-a7a2-e02e8dc9e40f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:52:53.366327 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68868b27-d397-428a-a7a2-e02e8dc9e40f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:52:53.367769 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68868b27-d397-428a-a7a2-e02e8dc9e40f service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:52:53.377456 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1252/2491] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:52:53 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:52:54.986991 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b99f34-3d15-4c26-af8b-77f1a104022b None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:52:54.987500 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b99f34-3d15-4c26-af8b-77f1a104022b None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:52:54.987600 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b99f34-3d15-4c26-af8b-77f1a104022b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:52:55.007430 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-d6b99f34-3d15-4c26-af8b-77f1a104022b None None] MFA Rules not processed for user `a30dc1dd2b00463bb3c7957864c56043`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:52:55.009107 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-d6b99f34-3d15-4c26-af8b-77f1a104022b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:52:55.017243 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1240/2492] 10.222.0.36 () {58 vars in 967 bytes} [Thu May 19 16:52:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:52:55.048043 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68868b27-d397-428a-a7a2-e02e8dc9e40f service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:52:55.060201 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68868b27-d397-428a-a7a2-e02e8dc9e40f service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:52:55.066217 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee57d2-7c28-48ba-8441-9313a780e848 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffe7cebc6f3b40d8a8e1866ed2da9822', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:52:55.067130 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee57d2-7c28-48ba-8441-9313a780e848 service swift] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:52:55.067653 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee57d2-7c28-48ba-8441-9313a780e848 service swift] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:52:55.068036 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee57d2-7c28-48ba-8441-9313a780e848 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:52:55.077566 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cee57d2-7c28-48ba-8441-9313a780e848 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:52:55.079350 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cee57d2-7c28-48ba-8441-9313a780e848 service swift] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:52:55.088442 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1253/2493] 10.222.0.36 () {60 vars in 1384 bytes} [Thu May 19 16:52:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 16:53:05.001292 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:53:05.084991 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=36 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:53:18.284752 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b99f34-3d15-4c26-af8b-77f1a104022b None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:18.286180 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-d6b99f34-3d15-4c26-af8b-77f1a104022b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:53:18.344143 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b99f34-3d15-4c26-af8b-77f1a104022b None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:18.347762 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2b8f96-66b6-4fa5-8e7e-725a99b61e8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:18.348434 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b8f96-66b6-4fa5-8e7e-725a99b61e8e service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:18.348738 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b8f96-66b6-4fa5-8e7e-725a99b61e8e service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:18.349542 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b8f96-66b6-4fa5-8e7e-725a99b61e8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:18.351317 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-3b2b8f96-66b6-4fa5-8e7e-725a99b61e8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:53:18.365069 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2b8f96-66b6-4fa5-8e7e-725a99b61e8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:18.366570 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2b8f96-66b6-4fa5-8e7e-725a99b61e8e service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:18.377689 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1241/2494] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:53:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:53:28.372767 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=26 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:53:47.348142 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee57d2-7c28-48ba-8441-9313a780e848 service swift] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:47.357323 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee57d2-7c28-48ba-8441-9313a780e848 service swift] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:47.360849 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da41dcc6-ce18-4a95-a238-9eee59ec003a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:47.361513 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da41dcc6-ce18-4a95-a238-9eee59ec003a service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:47.361801 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da41dcc6-ce18-4a95-a238-9eee59ec003a service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:47.362080 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da41dcc6-ce18-4a95-a238-9eee59ec003a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:47.363830 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-da41dcc6-ce18-4a95-a238-9eee59ec003a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:53:47.437515 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da41dcc6-ce18-4a95-a238-9eee59ec003a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:47.441464 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da41dcc6-ce18-4a95-a238-9eee59ec003a service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:47.453822 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1254/2495] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:53:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:53:50.565222 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2b8f96-66b6-4fa5-8e7e-725a99b61e8e service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:50.567795 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-3b2b8f96-66b6-4fa5-8e7e-725a99b61e8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:53:50.582249 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2b8f96-66b6-4fa5-8e7e-725a99b61e8e service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:50.587461 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa53902-39f5-429d-be99-663643c36e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:50.588334 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa53902-39f5-429d-be99-663643c36e8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:50.588652 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa53902-39f5-429d-be99-663643c36e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:50.588958 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa53902-39f5-429d-be99-663643c36e8e admin admin] PATH_INFO: `/v3/users/4077d35bf1044be79bfaa013e13c530e` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:50.590456 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa53902-39f5-429d-be99-663643c36e8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4077d35bf1044be79bfaa013e13c530e)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:50.592374 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa53902-39f5-429d-be99-663643c36e8e admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:50.643214 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-1fa53902-39f5-429d-be99-663643c36e8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4077d35bf1044be79bfaa013e13c530e'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:50.643885 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-1fa53902-39f5-429d-be99-663643c36e8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4077d35bf1044be79bfaa013e13c530e'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:50.651494 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-1fa53902-39f5-429d-be99-663643c36e8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4077d35bf1044be79bfaa013e13c530e'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:50.663165 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-1fa53902-39f5-429d-be99-663643c36e8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4077d35bf1044be79bfaa013e13c530e'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:50.663797 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-1fa53902-39f5-429d-be99-663643c36e8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4077d35bf1044be79bfaa013e13c530e'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:50.675872 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1242/2496] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:53:50 2022] DELETE /identity/v3/users/4077d35bf1044be79bfaa013e13c530e => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:53:50.847933 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da41dcc6-ce18-4a95-a238-9eee59ec003a service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:50.868228 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da41dcc6-ce18-4a95-a238-9eee59ec003a service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:50.932086 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d6f1d-315c-47ff-bcbf-5e6ebb47c5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:50.932964 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d6f1d-315c-47ff-bcbf-5e6ebb47c5a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:50.933282 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d6f1d-315c-47ff-bcbf-5e6ebb47c5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:50.933563 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d6f1d-315c-47ff-bcbf-5e6ebb47c5a9 admin admin] PATH_INFO: `/v3/projects/fd5d1150a9ed483bbd298b896bcf7f46` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:50.934920 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461d6f1d-315c-47ff-bcbf-5e6ebb47c5a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd5d1150a9ed483bbd298b896bcf7f46)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:50.936826 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461d6f1d-315c-47ff-bcbf-5e6ebb47c5a9 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:50.982353 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-461d6f1d-315c-47ff-bcbf-5e6ebb47c5a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd5d1150a9ed483bbd298b896bcf7f46', 'request_id': 'req-461d6f1d-315c-47ff-bcbf-5e6ebb47c5a9'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:50.987741 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-461d6f1d-315c-47ff-bcbf-5e6ebb47c5a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd5d1150a9ed483bbd298b896bcf7f46', 'request_id': 'req-461d6f1d-315c-47ff-bcbf-5e6ebb47c5a9'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:50.994337 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-461d6f1d-315c-47ff-bcbf-5e6ebb47c5a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd5d1150a9ed483bbd298b896bcf7f46', 'request_id': 'req-461d6f1d-315c-47ff-bcbf-5e6ebb47c5a9'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:51.007072 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-461d6f1d-315c-47ff-bcbf-5e6ebb47c5a9 admin admin] The token cache is being invalidate because project fd5d1150a9ed483bbd298b896bcf7f46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:53:51.007363 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-461d6f1d-315c-47ff-bcbf-5e6ebb47c5a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:51.007993 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-461d6f1d-315c-47ff-bcbf-5e6ebb47c5a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:51.009563 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1255/2497] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:53:50 2022] DELETE /identity/v3/projects/fd5d1150a9ed483bbd298b896bcf7f46 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:53:51.014707 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d31dd2-2756-449d-9da8-21ebdb797b15 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:51.015004 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d31dd2-2756-449d-9da8-21ebdb797b15 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:51.015289 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d31dd2-2756-449d-9da8-21ebdb797b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:51.033070 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-d3d31dd2-2756-449d-9da8-21ebdb797b15 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:53:51.034306 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-d3d31dd2-2756-449d-9da8-21ebdb797b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:53:51.092930 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1243/2498] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:53:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:53:51.101331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c749895-710f-46f5-96be-50f48ab579c4 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:51.101629 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c749895-710f-46f5-96be-50f48ab579c4 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:51.101913 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c749895-710f-46f5-96be-50f48ab579c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:51.120334 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-5c749895-710f-46f5-96be-50f48ab579c4 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:53:51.121568 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-5c749895-710f-46f5-96be-50f48ab579c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:53:51.129632 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1256/2499] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:53:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:53:51.134182 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d31dd2-2756-449d-9da8-21ebdb797b15 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:51.143607 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d31dd2-2756-449d-9da8-21ebdb797b15 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:51.148149 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c534c5-c7af-45f0-b420-e654fb6eed73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:51.148793 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c534c5-c7af-45f0-b420-e654fb6eed73 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:51.149079 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c534c5-c7af-45f0-b420-e654fb6eed73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:51.149394 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c534c5-c7af-45f0-b420-e654fb6eed73 admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:51.150119 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c534c5-c7af-45f0-b420-e654fb6eed73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:53:51.150542 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c534c5-c7af-45f0-b420-e654fb6eed73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:51.152160 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c534c5-c7af-45f0-b420-e654fb6eed73 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:51.163492 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1244/2500] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:53:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:53:51.167504 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c749895-710f-46f5-96be-50f48ab579c4 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:51.176549 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c749895-710f-46f5-96be-50f48ab579c4 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:51.182073 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3f15e9-97b3-4626-8b5e-7f8ff9eac11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:51.182734 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f15e9-97b3-4626-8b5e-7f8ff9eac11b admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:51.183021 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f15e9-97b3-4626-8b5e-7f8ff9eac11b admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:51.183300 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f15e9-97b3-4626-8b5e-7f8ff9eac11b admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:51.183996 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3f15e9-97b3-4626-8b5e-7f8ff9eac11b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:51.186055 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3f15e9-97b3-4626-8b5e-7f8ff9eac11b admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:51.200884 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:53:51.213172 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1257/2501] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:53:51 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:53:51.216980 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c534c5-c7af-45f0-b420-e654fb6eed73 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:51.226897 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c534c5-c7af-45f0-b420-e654fb6eed73 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:51.282089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd5089-6b1c-44ad-a295-6d543c0bf30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:51.282773 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd5089-6b1c-44ad-a295-6d543c0bf30b admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:51.283063 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd5089-6b1c-44ad-a295-6d543c0bf30b admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:51.283366 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd5089-6b1c-44ad-a295-6d543c0bf30b admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:51.284052 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cd5089-6b1c-44ad-a295-6d543c0bf30b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:51.286003 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cd5089-6b1c-44ad-a295-6d543c0bf30b admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:51.292977 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:53:51.309853 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1245/2502] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:53:51 2022] POST /identity/v3/users => generated 285 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:53:51.312484 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3f15e9-97b3-4626-8b5e-7f8ff9eac11b admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:51.322660 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3f15e9-97b3-4626-8b5e-7f8ff9eac11b admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:51.327233 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e434af-70f2-456a-9d49-f39e806071b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:51.327879 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e434af-70f2-456a-9d49-f39e806071b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:51.328164 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e434af-70f2-456a-9d49-f39e806071b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:51.328442 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e434af-70f2-456a-9d49-f39e806071b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:51.329194 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e434af-70f2-456a-9d49-f39e806071b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:51.330819 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e434af-70f2-456a-9d49-f39e806071b0 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:51.340873 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1258/2503] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:53:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:53:51.344575 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd5089-6b1c-44ad-a295-6d543c0bf30b admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:51.359289 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd5089-6b1c-44ad-a295-6d543c0bf30b admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:51.364169 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ceb9e-96f5-4cf4-9831-791a8e92c4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:51.364892 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ceb9e-96f5-4cf4-9831-791a8e92c4e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:51.365203 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ceb9e-96f5-4cf4-9831-791a8e92c4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:51.366040 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ceb9e-96f5-4cf4-9831-791a8e92c4e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:51.366769 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ceb9e-96f5-4cf4-9831-791a8e92c4e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:51.368354 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ceb9e-96f5-4cf4-9831-791a8e92c4e4 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:51.378723 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1246/2504] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:53:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:53:51.382756 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e434af-70f2-456a-9d49-f39e806071b0 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:51.392442 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e434af-70f2-456a-9d49-f39e806071b0 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:51.397234 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c7496e-8c77-43a6-8912-f459cd4868fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:51.398122 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c7496e-8c77-43a6-8912-f459cd4868fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:51.398394 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c7496e-8c77-43a6-8912-f459cd4868fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:51.398677 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c7496e-8c77-43a6-8912-f459cd4868fe admin admin] PATH_INFO: `/v3/projects/13e6a3a8f26948bb8054b251d408ede1/users/566ad15016bf47b99ea895a6d2b6bdcc/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:51.414263 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c7496e-8c77-43a6-8912-f459cd4868fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=13e6a3a8f26948bb8054b251d408ede1, user_id=566ad15016bf47b99ea895a6d2b6bdcc, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:51.416968 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c7496e-8c77-43a6-8912-f459cd4868fe admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:51.430651 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1259/2505] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:53:51 2022] PUT /identity/v3/projects/13e6a3a8f26948bb8054b251d408ede1/users/566ad15016bf47b99ea895a6d2b6bdcc/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:53:51.440525 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ab6a9-0345-4f85-880d-77c04caef040 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:51.440821 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ab6a9-0345-4f85-880d-77c04caef040 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:51.441104 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ab6a9-0345-4f85-880d-77c04caef040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:51.468734 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-754ab6a9-0345-4f85-880d-77c04caef040 None None] MFA Rules not processed for user `566ad15016bf47b99ea895a6d2b6bdcc`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:53:51.470025 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-754ab6a9-0345-4f85-880d-77c04caef040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:53:51.533545 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1247/2506] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:53:51 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:53:51.538575 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c7496e-8c77-43a6-8912-f459cd4868fe admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:51.599562 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c7496e-8c77-43a6-8912-f459cd4868fe admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:51.604465 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6842f802-b917-4a7c-ae2a-011a3b636907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:51.605195 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842f802-b917-4a7c-ae2a-011a3b636907 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:51.605419 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842f802-b917-4a7c-ae2a-011a3b636907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:51.605841 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842f802-b917-4a7c-ae2a-011a3b636907 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:51.606674 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6842f802-b917-4a7c-ae2a-011a3b636907 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:51.608804 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6842f802-b917-4a7c-ae2a-011a3b636907 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:51.637220 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1260/2507] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:53:51 2022] POST /identity/v3/projects => generated 367 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:53:51.638845 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754ab6a9-0345-4f85-880d-77c04caef040 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:51.649920 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754ab6a9-0345-4f85-880d-77c04caef040 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:51.704652 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35dd3cb-bdf7-4b99-be69-a99b298eb4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:51.705354 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35dd3cb-bdf7-4b99-be69-a99b298eb4a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:51.705643 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35dd3cb-bdf7-4b99-be69-a99b298eb4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:51.705924 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35dd3cb-bdf7-4b99-be69-a99b298eb4a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:51.706628 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35dd3cb-bdf7-4b99-be69-a99b298eb4a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:51.708554 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35dd3cb-bdf7-4b99-be69-a99b298eb4a1 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:51.730495 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1248/2508] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:53:51 2022] POST /identity/v3/users => generated 290 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:53:51.731627 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6842f802-b917-4a7c-ae2a-011a3b636907 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:51.742260 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6842f802-b917-4a7c-ae2a-011a3b636907 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:51.747061 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7156eefb-6d01-4282-b965-8c962e92add2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:51.747729 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156eefb-6d01-4282-b965-8c962e92add2 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:51.747991 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156eefb-6d01-4282-b965-8c962e92add2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:51.748265 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156eefb-6d01-4282-b965-8c962e92add2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:51.748987 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7156eefb-6d01-4282-b965-8c962e92add2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:51.750621 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7156eefb-6d01-4282-b965-8c962e92add2 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:51.761523 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1261/2509] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:53:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:53:51.765379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35dd3cb-bdf7-4b99-be69-a99b298eb4a1 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:51.774954 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35dd3cb-bdf7-4b99-be69-a99b298eb4a1 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:51.779474 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5993d2da-6ac9-422c-9e26-ccd61036c7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:51.780333 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5993d2da-6ac9-422c-9e26-ccd61036c7ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:51.780633 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5993d2da-6ac9-422c-9e26-ccd61036c7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:51.780890 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5993d2da-6ac9-422c-9e26-ccd61036c7ff admin admin] PATH_INFO: `/v3/projects/6d3fde6415f94e41a9cfc8f793bc6c9e/users/da0ff66b1cc64bac9349d3f7e52a6b70/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:51.796007 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5993d2da-6ac9-422c-9e26-ccd61036c7ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d3fde6415f94e41a9cfc8f793bc6c9e, user_id=da0ff66b1cc64bac9349d3f7e52a6b70, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:51.798768 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5993d2da-6ac9-422c-9e26-ccd61036c7ff admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:51.811642 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1249/2510] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:53:51 2022] PUT /identity/v3/projects/6d3fde6415f94e41a9cfc8f793bc6c9e/users/da0ff66b1cc64bac9349d3f7e52a6b70/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:53:51.816372 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7156eefb-6d01-4282-b965-8c962e92add2 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:51.826987 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7156eefb-6d01-4282-b965-8c962e92add2 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:51.883146 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda928d-29d1-47fe-a3a0-ca30ce8b8a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:51.883811 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda928d-29d1-47fe-a3a0-ca30ce8b8a79 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:51.884098 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda928d-29d1-47fe-a3a0-ca30ce8b8a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:51.884376 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda928d-29d1-47fe-a3a0-ca30ce8b8a79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:51.885100 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fda928d-29d1-47fe-a3a0-ca30ce8b8a79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:51.886773 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fda928d-29d1-47fe-a3a0-ca30ce8b8a79 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:51.896702 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1262/2511] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:53:51 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:53:51.900830 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5993d2da-6ac9-422c-9e26-ccd61036c7ff admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:51.910998 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5993d2da-6ac9-422c-9e26-ccd61036c7ff admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:51.915785 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661844f1-5099-477f-989a-3226a4025568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:51.916668 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661844f1-5099-477f-989a-3226a4025568 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:51.916955 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661844f1-5099-477f-989a-3226a4025568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:51.917265 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661844f1-5099-477f-989a-3226a4025568 admin admin] PATH_INFO: `/v3/projects/6d3fde6415f94e41a9cfc8f793bc6c9e/users/da0ff66b1cc64bac9349d3f7e52a6b70/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:51.919218 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661844f1-5099-477f-989a-3226a4025568 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d3fde6415f94e41a9cfc8f793bc6c9e, user_id=da0ff66b1cc64bac9349d3f7e52a6b70, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:51.921911 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661844f1-5099-477f-989a-3226a4025568 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:51.932921 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1250/2512] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:53:51 2022] PUT /identity/v3/projects/6d3fde6415f94e41a9cfc8f793bc6c9e/users/da0ff66b1cc64bac9349d3f7e52a6b70/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:53:51.942931 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b095a0-ff1b-434c-ba7d-5fb5c139198a None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:51.943230 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b095a0-ff1b-434c-ba7d-5fb5c139198a None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:51.943604 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b095a0-ff1b-434c-ba7d-5fb5c139198a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:51.969682 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-a7b095a0-ff1b-434c-ba7d-5fb5c139198a None None] MFA Rules not processed for user `da0ff66b1cc64bac9349d3f7e52a6b70`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:53:51.970886 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-a7b095a0-ff1b-434c-ba7d-5fb5c139198a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:53:51.988896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:53:52.041159 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1263/2513] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:53:51 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:53:52.052219 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661844f1-5099-477f-989a-3226a4025568 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:52.053980 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-661844f1-5099-477f-989a-3226a4025568 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:53:52.076454 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:53:52.115159 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661844f1-5099-477f-989a-3226a4025568 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:52.118838 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3177dd5-f47c-4d38-b220-f462ea48b425 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:52.119499 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3177dd5-f47c-4d38-b220-f462ea48b425 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:52.119769 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3177dd5-f47c-4d38-b220-f462ea48b425 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:52.120049 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3177dd5-f47c-4d38-b220-f462ea48b425 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:52.129260 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3177dd5-f47c-4d38-b220-f462ea48b425 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:52.130693 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3177dd5-f47c-4d38-b220-f462ea48b425 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:52.140529 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1251/2514] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:53:52 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:53:52.185342 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b095a0-ff1b-434c-ba7d-5fb5c139198a None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:52.186722 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-a7b095a0-ff1b-434c-ba7d-5fb5c139198a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:53:52.199654 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b095a0-ff1b-434c-ba7d-5fb5c139198a None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:52.203305 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73728904-bb92-4227-a54b-e6ee7d679a39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:52.203953 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73728904-bb92-4227-a54b-e6ee7d679a39 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:52.204249 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73728904-bb92-4227-a54b-e6ee7d679a39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:52.204528 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73728904-bb92-4227-a54b-e6ee7d679a39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:52.260707 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73728904-bb92-4227-a54b-e6ee7d679a39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:52.262184 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73728904-bb92-4227-a54b-e6ee7d679a39 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:52.270875 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1264/2515] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:53:52 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:53:55.510470 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3177dd5-f47c-4d38-b220-f462ea48b425 service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:55.571024 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3177dd5-f47c-4d38-b220-f462ea48b425 service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:55.575478 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de10fc6e-c610-42eb-89f8-0b78154372ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:55.576341 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de10fc6e-c610-42eb-89f8-0b78154372ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:55.576624 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de10fc6e-c610-42eb-89f8-0b78154372ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:55.576903 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de10fc6e-c610-42eb-89f8-0b78154372ef admin admin] PATH_INFO: `/v3/users/566ad15016bf47b99ea895a6d2b6bdcc` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:55.578115 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de10fc6e-c610-42eb-89f8-0b78154372ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=566ad15016bf47b99ea895a6d2b6bdcc)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:55.580034 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de10fc6e-c610-42eb-89f8-0b78154372ef admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:55.625681 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-de10fc6e-c610-42eb-89f8-0b78154372ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '566ad15016bf47b99ea895a6d2b6bdcc'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:55.626329 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-de10fc6e-c610-42eb-89f8-0b78154372ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '566ad15016bf47b99ea895a6d2b6bdcc'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:55.632743 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-de10fc6e-c610-42eb-89f8-0b78154372ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '566ad15016bf47b99ea895a6d2b6bdcc'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:55.644343 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-de10fc6e-c610-42eb-89f8-0b78154372ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '566ad15016bf47b99ea895a6d2b6bdcc'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:55.644971 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-de10fc6e-c610-42eb-89f8-0b78154372ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '566ad15016bf47b99ea895a6d2b6bdcc'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:55.657327 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1252/2516] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:53:55 2022] DELETE /identity/v3/users/566ad15016bf47b99ea895a6d2b6bdcc => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:53:55.661802 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586274e6-5677-4ae1-bc23-fc262afbc64c None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:55.662844 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586274e6-5677-4ae1-bc23-fc262afbc64c None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:55.662974 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586274e6-5677-4ae1-bc23-fc262afbc64c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:55.679397 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-586274e6-5677-4ae1-bc23-fc262afbc64c None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:53:55.680745 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-586274e6-5677-4ae1-bc23-fc262afbc64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:53:55.738618 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1265/2517] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:53:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:53:55.748383 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de10fc6e-c610-42eb-89f8-0b78154372ef admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:55.749805 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-de10fc6e-c610-42eb-89f8-0b78154372ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:53:55.795895 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:53:55.808779 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de10fc6e-c610-42eb-89f8-0b78154372ef admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:55.812416 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aae564-dc89-4b06-a284-86f527c3f5d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:55.813061 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aae564-dc89-4b06-a284-86f527c3f5d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:55.813378 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aae564-dc89-4b06-a284-86f527c3f5d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:55.813791 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aae564-dc89-4b06-a284-86f527c3f5d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:55.823318 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aae564-dc89-4b06-a284-86f527c3f5d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:55.824692 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aae564-dc89-4b06-a284-86f527c3f5d7 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:55.833929 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1253/2518] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:53:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:53:56.100762 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586274e6-5677-4ae1-bc23-fc262afbc64c None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:56.102371 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-586274e6-5677-4ae1-bc23-fc262afbc64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:53:56.116083 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586274e6-5677-4ae1-bc23-fc262afbc64c None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:56.120799 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efefbcbf-7286-4c02-9a2f-5ad03605098a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:56.121966 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efefbcbf-7286-4c02-9a2f-5ad03605098a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:56.122256 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efefbcbf-7286-4c02-9a2f-5ad03605098a admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:56.122536 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efefbcbf-7286-4c02-9a2f-5ad03605098a admin admin] PATH_INFO: `/v3/projects/13e6a3a8f26948bb8054b251d408ede1` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:56.123742 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efefbcbf-7286-4c02-9a2f-5ad03605098a admin admin] RBAC: Authorizing `identity:delete_project(project_id=13e6a3a8f26948bb8054b251d408ede1)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:56.125717 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efefbcbf-7286-4c02-9a2f-5ad03605098a admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:56.173957 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-efefbcbf-7286-4c02-9a2f-5ad03605098a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13e6a3a8f26948bb8054b251d408ede1', 'request_id': 'req-efefbcbf-7286-4c02-9a2f-5ad03605098a'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:56.175323 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:53:56.182148 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-efefbcbf-7286-4c02-9a2f-5ad03605098a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13e6a3a8f26948bb8054b251d408ede1', 'request_id': 'req-efefbcbf-7286-4c02-9a2f-5ad03605098a'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:56.188496 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-efefbcbf-7286-4c02-9a2f-5ad03605098a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13e6a3a8f26948bb8054b251d408ede1', 'request_id': 'req-efefbcbf-7286-4c02-9a2f-5ad03605098a'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:56.201080 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-efefbcbf-7286-4c02-9a2f-5ad03605098a admin admin] The token cache is being invalidate because project 13e6a3a8f26948bb8054b251d408ede1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:53:56.201399 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-efefbcbf-7286-4c02-9a2f-5ad03605098a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:56.202015 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-efefbcbf-7286-4c02-9a2f-5ad03605098a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:56.203586 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1266/2519] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:53:56 2022] DELETE /identity/v3/projects/13e6a3a8f26948bb8054b251d408ede1 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:53:56.207899 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aae564-dc89-4b06-a284-86f527c3f5d7 service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:56.217492 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aae564-dc89-4b06-a284-86f527c3f5d7 service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:56.274334 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35548493-e331-4520-b0f3-2fff172bbab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:56.275213 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35548493-e331-4520-b0f3-2fff172bbab4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:56.275506 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35548493-e331-4520-b0f3-2fff172bbab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:56.275790 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35548493-e331-4520-b0f3-2fff172bbab4 admin admin] PATH_INFO: `/v3/users/da0ff66b1cc64bac9349d3f7e52a6b70` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:56.276992 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35548493-e331-4520-b0f3-2fff172bbab4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da0ff66b1cc64bac9349d3f7e52a6b70)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:56.278976 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35548493-e331-4520-b0f3-2fff172bbab4 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:56.324374 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-35548493-e331-4520-b0f3-2fff172bbab4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da0ff66b1cc64bac9349d3f7e52a6b70'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:56.325018 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-35548493-e331-4520-b0f3-2fff172bbab4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da0ff66b1cc64bac9349d3f7e52a6b70'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:56.331782 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-35548493-e331-4520-b0f3-2fff172bbab4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da0ff66b1cc64bac9349d3f7e52a6b70'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:56.343599 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-35548493-e331-4520-b0f3-2fff172bbab4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da0ff66b1cc64bac9349d3f7e52a6b70'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:56.344170 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-35548493-e331-4520-b0f3-2fff172bbab4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da0ff66b1cc64bac9349d3f7e52a6b70'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:56.356021 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1254/2520] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:53:56 2022] DELETE /identity/v3/users/da0ff66b1cc64bac9349d3f7e52a6b70 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:53:56.638086 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efefbcbf-7286-4c02-9a2f-5ad03605098a admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:56.639622 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-efefbcbf-7286-4c02-9a2f-5ad03605098a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:53:56.654445 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efefbcbf-7286-4c02-9a2f-5ad03605098a admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:56.713174 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d23ebe-bf69-4158-a83d-501fad0f3ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:56.714061 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d23ebe-bf69-4158-a83d-501fad0f3ac2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:56.714352 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d23ebe-bf69-4158-a83d-501fad0f3ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:56.714637 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d23ebe-bf69-4158-a83d-501fad0f3ac2 admin admin] PATH_INFO: `/v3/projects/6d3fde6415f94e41a9cfc8f793bc6c9e` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:56.715823 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d23ebe-bf69-4158-a83d-501fad0f3ac2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d3fde6415f94e41a9cfc8f793bc6c9e)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:56.717800 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d23ebe-bf69-4158-a83d-501fad0f3ac2 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:56.764273 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a8d23ebe-bf69-4158-a83d-501fad0f3ac2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d3fde6415f94e41a9cfc8f793bc6c9e', 'request_id': 'req-a8d23ebe-bf69-4158-a83d-501fad0f3ac2'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:56.769920 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a8d23ebe-bf69-4158-a83d-501fad0f3ac2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d3fde6415f94e41a9cfc8f793bc6c9e', 'request_id': 'req-a8d23ebe-bf69-4158-a83d-501fad0f3ac2'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:56.776510 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a8d23ebe-bf69-4158-a83d-501fad0f3ac2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d3fde6415f94e41a9cfc8f793bc6c9e', 'request_id': 'req-a8d23ebe-bf69-4158-a83d-501fad0f3ac2'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:56.788912 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a8d23ebe-bf69-4158-a83d-501fad0f3ac2 admin admin] The token cache is being invalidate because project 6d3fde6415f94e41a9cfc8f793bc6c9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:53:56.789230 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a8d23ebe-bf69-4158-a83d-501fad0f3ac2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:56.789856 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a8d23ebe-bf69-4158-a83d-501fad0f3ac2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:56.791397 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1267/2521] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:53:56 2022] DELETE /identity/v3/projects/6d3fde6415f94e41a9cfc8f793bc6c9e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:53:56.796286 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a5716-6346-4e0e-a711-d4979d744667 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:56.796739 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a5716-6346-4e0e-a711-d4979d744667 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:56.796862 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a5716-6346-4e0e-a711-d4979d744667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:56.817234 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-905a5716-6346-4e0e-a711-d4979d744667 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:53:56.819073 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-905a5716-6346-4e0e-a711-d4979d744667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:53:56.875092 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1255/2522] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:53:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:53:56.884487 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d369db4c-279e-4b2a-923b-0383e7230c1c None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:56.884784 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d369db4c-279e-4b2a-923b-0383e7230c1c None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:56.885093 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d369db4c-279e-4b2a-923b-0383e7230c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:56.903883 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-d369db4c-279e-4b2a-923b-0383e7230c1c None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:53:56.905087 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d369db4c-279e-4b2a-923b-0383e7230c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:53:56.913505 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1268/2523] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:53:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:53:56.917721 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905a5716-6346-4e0e-a711-d4979d744667 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:56.923908 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:53:56.928577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905a5716-6346-4e0e-a711-d4979d744667 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:56.934724 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36806d6-f31b-4bdd-bd30-f2c638d49109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:56.935421 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36806d6-f31b-4bdd-bd30-f2c638d49109 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:56.935903 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36806d6-f31b-4bdd-bd30-f2c638d49109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:56.936210 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36806d6-f31b-4bdd-bd30-f2c638d49109 admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:56.937105 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36806d6-f31b-4bdd-bd30-f2c638d49109 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:53:56.937632 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36806d6-f31b-4bdd-bd30-f2c638d49109 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:56.939265 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36806d6-f31b-4bdd-bd30-f2c638d49109 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:56.950992 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1256/2524] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:53:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:53:56.955428 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d369db4c-279e-4b2a-923b-0383e7230c1c None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:56.965236 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d369db4c-279e-4b2a-923b-0383e7230c1c None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:56.969943 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5764376a-2550-467d-8962-828ba0ba8b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:56.970596 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764376a-2550-467d-8962-828ba0ba8b0e admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:56.970888 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764376a-2550-467d-8962-828ba0ba8b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:56.971170 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764376a-2550-467d-8962-828ba0ba8b0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:56.971880 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5764376a-2550-467d-8962-828ba0ba8b0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:56.973852 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5764376a-2550-467d-8962-828ba0ba8b0e admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:57.000184 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1269/2525] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:53:56 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:53:57.004341 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36806d6-f31b-4bdd-bd30-f2c638d49109 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:57.014775 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36806d6-f31b-4bdd-bd30-f2c638d49109 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:57.070166 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8482c8-1513-41f0-9128-2c2460a05c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:57.070833 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8482c8-1513-41f0-9128-2c2460a05c6e admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:57.071125 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8482c8-1513-41f0-9128-2c2460a05c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:57.071409 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8482c8-1513-41f0-9128-2c2460a05c6e admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:57.072098 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8482c8-1513-41f0-9128-2c2460a05c6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:57.074053 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8482c8-1513-41f0-9128-2c2460a05c6e admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:57.093724 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1257/2526] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:53:57 2022] POST /identity/v3/users => generated 294 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:53:57.097914 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5764376a-2550-467d-8962-828ba0ba8b0e admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:57.108830 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5764376a-2550-467d-8962-828ba0ba8b0e admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:57.113623 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17ec608-f987-42b6-8d90-abd541b16435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:57.114280 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ec608-f987-42b6-8d90-abd541b16435 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:57.114571 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ec608-f987-42b6-8d90-abd541b16435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:57.114852 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ec608-f987-42b6-8d90-abd541b16435 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:57.115615 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17ec608-f987-42b6-8d90-abd541b16435 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:57.117513 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17ec608-f987-42b6-8d90-abd541b16435 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:57.129215 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1270/2527] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:53:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:53:57.133131 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8482c8-1513-41f0-9128-2c2460a05c6e admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:57.142735 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8482c8-1513-41f0-9128-2c2460a05c6e admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:57.147821 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76960693-44c7-4b08-b55a-a10be07c3cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:57.148464 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76960693-44c7-4b08-b55a-a10be07c3cdd admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:57.148750 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76960693-44c7-4b08-b55a-a10be07c3cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:57.149029 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76960693-44c7-4b08-b55a-a10be07c3cdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:57.149787 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76960693-44c7-4b08-b55a-a10be07c3cdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:57.151389 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76960693-44c7-4b08-b55a-a10be07c3cdd admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:57.160987 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1258/2528] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:53:57 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:53:57.165678 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17ec608-f987-42b6-8d90-abd541b16435 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:57.175484 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17ec608-f987-42b6-8d90-abd541b16435 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:57.180568 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8844c1a4-8d87-4884-8722-52d966cde6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:57.181461 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844c1a4-8d87-4884-8722-52d966cde6ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:57.181745 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844c1a4-8d87-4884-8722-52d966cde6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:57.182021 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844c1a4-8d87-4884-8722-52d966cde6ec admin admin] PATH_INFO: `/v3/projects/cb864376a4684b7f89bcbbb1257a34b3/users/9ff17e3c9bac48e9820dde1ac0389bb3/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:57.196718 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8844c1a4-8d87-4884-8722-52d966cde6ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb864376a4684b7f89bcbbb1257a34b3, user_id=9ff17e3c9bac48e9820dde1ac0389bb3, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:57.199388 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8844c1a4-8d87-4884-8722-52d966cde6ec admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:57.212049 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1271/2529] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:53:57 2022] PUT /identity/v3/projects/cb864376a4684b7f89bcbbb1257a34b3/users/9ff17e3c9bac48e9820dde1ac0389bb3/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:53:57.222073 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864e8e6d-aba9-4cf7-8d92-530cc3474d15 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:57.222399 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864e8e6d-aba9-4cf7-8d92-530cc3474d15 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:57.222684 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864e8e6d-aba9-4cf7-8d92-530cc3474d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:57.249728 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-864e8e6d-aba9-4cf7-8d92-530cc3474d15 None None] MFA Rules not processed for user `9ff17e3c9bac48e9820dde1ac0389bb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:53:57.250929 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-864e8e6d-aba9-4cf7-8d92-530cc3474d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:53:57.316284 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1259/2530] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:53:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:53:57.321556 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8844c1a4-8d87-4884-8722-52d966cde6ec admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:57.327112 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:53:57.387954 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8844c1a4-8d87-4884-8722-52d966cde6ec admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:57.392513 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da490e9-2255-403c-b645-9571c159edf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:57.393175 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da490e9-2255-403c-b645-9571c159edf6 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:57.393474 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da490e9-2255-403c-b645-9571c159edf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:57.393748 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da490e9-2255-403c-b645-9571c159edf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:57.394430 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da490e9-2255-403c-b645-9571c159edf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:57.396396 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da490e9-2255-403c-b645-9571c159edf6 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:57.421342 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1272/2531] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:53:57 2022] POST /identity/v3/projects => generated 387 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:53:57.425418 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864e8e6d-aba9-4cf7-8d92-530cc3474d15 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:57.436313 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864e8e6d-aba9-4cf7-8d92-530cc3474d15 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:57.491973 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ff0d85-e365-44b7-8e65-32b03675e615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:57.492642 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff0d85-e365-44b7-8e65-32b03675e615 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:57.492931 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff0d85-e365-44b7-8e65-32b03675e615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:57.493244 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff0d85-e365-44b7-8e65-32b03675e615 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:57.493946 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ff0d85-e365-44b7-8e65-32b03675e615 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:57.495858 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ff0d85-e365-44b7-8e65-32b03675e615 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:57.502652 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:53:57.517030 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1260/2532] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:53:57 2022] POST /identity/v3/users => generated 300 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:53:57.520920 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da490e9-2255-403c-b645-9571c159edf6 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:57.531805 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da490e9-2255-403c-b645-9571c159edf6 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:57.536445 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221f99b4-8cdb-4c17-92cc-d360f5e5bc19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:57.537105 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f99b4-8cdb-4c17-92cc-d360f5e5bc19 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:57.537446 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f99b4-8cdb-4c17-92cc-d360f5e5bc19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:57.537728 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f99b4-8cdb-4c17-92cc-d360f5e5bc19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:57.538461 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221f99b4-8cdb-4c17-92cc-d360f5e5bc19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:57.540069 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221f99b4-8cdb-4c17-92cc-d360f5e5bc19 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:57.550842 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1273/2533] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:53:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:53:57.554905 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ff0d85-e365-44b7-8e65-32b03675e615 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:57.564658 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ff0d85-e365-44b7-8e65-32b03675e615 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:57.569771 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1616207-ce96-4fb3-b127-c1d7dcfa7ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:57.570646 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1616207-ce96-4fb3-b127-c1d7dcfa7ed7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:57.570932 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1616207-ce96-4fb3-b127-c1d7dcfa7ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:57.571211 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1616207-ce96-4fb3-b127-c1d7dcfa7ed7 admin admin] PATH_INFO: `/v3/projects/504cce2301a245d9935c44038a1d021e/users/418f82ee063a4f218a6de99412d9f880/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:57.586118 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1616207-ce96-4fb3-b127-c1d7dcfa7ed7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=504cce2301a245d9935c44038a1d021e, user_id=418f82ee063a4f218a6de99412d9f880, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:57.588789 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1616207-ce96-4fb3-b127-c1d7dcfa7ed7 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:57.601466 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1261/2534] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:53:57 2022] PUT /identity/v3/projects/504cce2301a245d9935c44038a1d021e/users/418f82ee063a4f218a6de99412d9f880/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:53:57.605331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221f99b4-8cdb-4c17-92cc-d360f5e5bc19 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:57.615716 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221f99b4-8cdb-4c17-92cc-d360f5e5bc19 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:57.671683 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7cde88-f7bf-4811-bc14-88bf5828212a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:57.672280 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7cde88-f7bf-4811-bc14-88bf5828212a admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:57.672571 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7cde88-f7bf-4811-bc14-88bf5828212a admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:57.672854 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7cde88-f7bf-4811-bc14-88bf5828212a admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:57.673628 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7cde88-f7bf-4811-bc14-88bf5828212a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:57.675243 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7cde88-f7bf-4811-bc14-88bf5828212a admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:57.685365 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1274/2535] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:53:57 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:53:57.689292 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1616207-ce96-4fb3-b127-c1d7dcfa7ed7 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:57.699941 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1616207-ce96-4fb3-b127-c1d7dcfa7ed7 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:57.704557 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7994cf4f-2703-48c5-ad37-d13af225d937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:57.705470 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994cf4f-2703-48c5-ad37-d13af225d937 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:57.705757 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994cf4f-2703-48c5-ad37-d13af225d937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:57.706039 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994cf4f-2703-48c5-ad37-d13af225d937 admin admin] PATH_INFO: `/v3/projects/504cce2301a245d9935c44038a1d021e/users/418f82ee063a4f218a6de99412d9f880/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:57.707978 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7994cf4f-2703-48c5-ad37-d13af225d937 admin admin] RBAC: Authorizing `identity:create_grant(project_id=504cce2301a245d9935c44038a1d021e, user_id=418f82ee063a4f218a6de99412d9f880, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:57.710628 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7994cf4f-2703-48c5-ad37-d13af225d937 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:57.720901 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1262/2536] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:53:57 2022] PUT /identity/v3/projects/504cce2301a245d9935c44038a1d021e/users/418f82ee063a4f218a6de99412d9f880/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:53:57.730664 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f631cb-867f-412f-a00e-5a976e80f579 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:57.731135 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f631cb-867f-412f-a00e-5a976e80f579 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:57.731251 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f631cb-867f-412f-a00e-5a976e80f579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:57.757659 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-63f631cb-867f-412f-a00e-5a976e80f579 None None] MFA Rules not processed for user `418f82ee063a4f218a6de99412d9f880`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:53:57.758841 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-63f631cb-867f-412f-a00e-5a976e80f579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:53:57.822736 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:53:57.826906 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1275/2537] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:53:57 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:53:57.838963 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7994cf4f-2703-48c5-ad37-d13af225d937 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:57.840835 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-7994cf4f-2703-48c5-ad37-d13af225d937 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:53:57.899689 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7994cf4f-2703-48c5-ad37-d13af225d937 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:57.903362 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52cdce9-9f6b-4e89-8272-0cdf98dc7b38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:57.904012 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52cdce9-9f6b-4e89-8272-0cdf98dc7b38 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:57.904302 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52cdce9-9f6b-4e89-8272-0cdf98dc7b38 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:57.904584 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52cdce9-9f6b-4e89-8272-0cdf98dc7b38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:57.913733 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52cdce9-9f6b-4e89-8272-0cdf98dc7b38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:57.915173 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52cdce9-9f6b-4e89-8272-0cdf98dc7b38 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:57.924706 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1263/2538] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:53:57 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:53:57.988267 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f631cb-867f-412f-a00e-5a976e80f579 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:58.049517 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f631cb-867f-412f-a00e-5a976e80f579 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:58.054770 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7d305d-5c99-44be-a13d-b5136f2e80fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:58.055669 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d305d-5c99-44be-a13d-b5136f2e80fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:58.055957 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d305d-5c99-44be-a13d-b5136f2e80fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:58.056238 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d305d-5c99-44be-a13d-b5136f2e80fe admin admin] PATH_INFO: `/v3/users/9ff17e3c9bac48e9820dde1ac0389bb3` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:58.057484 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7d305d-5c99-44be-a13d-b5136f2e80fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ff17e3c9bac48e9820dde1ac0389bb3)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:58.059422 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7d305d-5c99-44be-a13d-b5136f2e80fe admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:58.103941 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-0e7d305d-5c99-44be-a13d-b5136f2e80fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ff17e3c9bac48e9820dde1ac0389bb3'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:58.104620 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-0e7d305d-5c99-44be-a13d-b5136f2e80fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ff17e3c9bac48e9820dde1ac0389bb3'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:58.105267 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-0e7d305d-5c99-44be-a13d-b5136f2e80fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ff17e3c9bac48e9820dde1ac0389bb3'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:58.112851 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-0e7d305d-5c99-44be-a13d-b5136f2e80fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ff17e3c9bac48e9820dde1ac0389bb3'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:58.125678 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-0e7d305d-5c99-44be-a13d-b5136f2e80fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ff17e3c9bac48e9820dde1ac0389bb3'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:58.137558 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1276/2539] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:53:57 2022] DELETE /identity/v3/users/9ff17e3c9bac48e9820dde1ac0389bb3 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:53:58.142008 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb64d9a-b4b0-45a4-a159-16226f316524 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:58.142312 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb64d9a-b4b0-45a4-a159-16226f316524 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:58.142602 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb64d9a-b4b0-45a4-a159-16226f316524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:58.159859 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-cbb64d9a-b4b0-45a4-a159-16226f316524 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:53:58.161058 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-cbb64d9a-b4b0-45a4-a159-16226f316524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:53:58.218746 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1264/2540] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:53:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:53:58.229540 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7d305d-5c99-44be-a13d-b5136f2e80fe admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:58.230945 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-0e7d305d-5c99-44be-a13d-b5136f2e80fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:53:58.287891 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7d305d-5c99-44be-a13d-b5136f2e80fe admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:58.291654 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd4ef2b-e5de-402f-aba4-7518b6939acb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:58.292299 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd4ef2b-e5de-402f-aba4-7518b6939acb service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:58.292587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd4ef2b-e5de-402f-aba4-7518b6939acb service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:58.292868 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd4ef2b-e5de-402f-aba4-7518b6939acb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:58.302019 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd4ef2b-e5de-402f-aba4-7518b6939acb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:58.304088 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd4ef2b-e5de-402f-aba4-7518b6939acb service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:58.313513 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1277/2541] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:53:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:53:58.578451 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb64d9a-b4b0-45a4-a159-16226f316524 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:58.579889 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-cbb64d9a-b4b0-45a4-a159-16226f316524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:53:58.595611 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb64d9a-b4b0-45a4-a159-16226f316524 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:58.600442 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571adf1c-b858-4c5f-af71-0902c5b24af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:58.601482 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571adf1c-b858-4c5f-af71-0902c5b24af9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:58.601769 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571adf1c-b858-4c5f-af71-0902c5b24af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:58.602048 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571adf1c-b858-4c5f-af71-0902c5b24af9 admin admin] PATH_INFO: `/v3/projects/cb864376a4684b7f89bcbbb1257a34b3` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:58.603786 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571adf1c-b858-4c5f-af71-0902c5b24af9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb864376a4684b7f89bcbbb1257a34b3)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:58.605725 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571adf1c-b858-4c5f-af71-0902c5b24af9 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:58.651406 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-571adf1c-b858-4c5f-af71-0902c5b24af9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb864376a4684b7f89bcbbb1257a34b3', 'request_id': 'req-571adf1c-b858-4c5f-af71-0902c5b24af9'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:58.652591 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:INSERT=5,keystone:SELECT=91,keystone:DELETE=4 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:53:58.657950 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-571adf1c-b858-4c5f-af71-0902c5b24af9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb864376a4684b7f89bcbbb1257a34b3', 'request_id': 'req-571adf1c-b858-4c5f-af71-0902c5b24af9'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:58.666165 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-571adf1c-b858-4c5f-af71-0902c5b24af9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb864376a4684b7f89bcbbb1257a34b3', 'request_id': 'req-571adf1c-b858-4c5f-af71-0902c5b24af9'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:58.677762 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-571adf1c-b858-4c5f-af71-0902c5b24af9 admin admin] The token cache is being invalidate because project cb864376a4684b7f89bcbbb1257a34b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:53:58.678056 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-571adf1c-b858-4c5f-af71-0902c5b24af9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:58.678705 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-571adf1c-b858-4c5f-af71-0902c5b24af9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:58.680211 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1265/2542] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:53:58 2022] DELETE /identity/v3/projects/cb864376a4684b7f89bcbbb1257a34b3 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:53:58.684238 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd4ef2b-e5de-402f-aba4-7518b6939acb service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:58.693779 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd4ef2b-e5de-402f-aba4-7518b6939acb service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:58.712796 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:53:58.752450 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8717a98-131b-45ef-91d2-f04a3bb60979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:58.753282 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8717a98-131b-45ef-91d2-f04a3bb60979 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:58.753574 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8717a98-131b-45ef-91d2-f04a3bb60979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:58.753857 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8717a98-131b-45ef-91d2-f04a3bb60979 admin admin] PATH_INFO: `/v3/users/418f82ee063a4f218a6de99412d9f880` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:58.755050 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8717a98-131b-45ef-91d2-f04a3bb60979 admin admin] RBAC: Authorizing `identity:delete_user(user_id=418f82ee063a4f218a6de99412d9f880)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:58.756983 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8717a98-131b-45ef-91d2-f04a3bb60979 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:58.803095 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-e8717a98-131b-45ef-91d2-f04a3bb60979 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '418f82ee063a4f218a6de99412d9f880'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:58.803738 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-e8717a98-131b-45ef-91d2-f04a3bb60979 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '418f82ee063a4f218a6de99412d9f880'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:58.804332 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-e8717a98-131b-45ef-91d2-f04a3bb60979 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '418f82ee063a4f218a6de99412d9f880'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:58.811738 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-e8717a98-131b-45ef-91d2-f04a3bb60979 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '418f82ee063a4f218a6de99412d9f880'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:58.824625 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-e8717a98-131b-45ef-91d2-f04a3bb60979 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '418f82ee063a4f218a6de99412d9f880'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:58.838977 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1278/2543] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:53:58 2022] DELETE /identity/v3/users/418f82ee063a4f218a6de99412d9f880 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:53:59.107655 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571adf1c-b858-4c5f-af71-0902c5b24af9 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:59.109118 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-571adf1c-b858-4c5f-af71-0902c5b24af9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:53:59.122570 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571adf1c-b858-4c5f-af71-0902c5b24af9 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:59.186481 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383235de-84d7-43d4-9f3f-48c8a61ea5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:59.187346 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383235de-84d7-43d4-9f3f-48c8a61ea5f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:59.187630 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383235de-84d7-43d4-9f3f-48c8a61ea5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:59.187910 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383235de-84d7-43d4-9f3f-48c8a61ea5f3 admin admin] PATH_INFO: `/v3/projects/504cce2301a245d9935c44038a1d021e` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:59.189098 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383235de-84d7-43d4-9f3f-48c8a61ea5f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=504cce2301a245d9935c44038a1d021e)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:59.191037 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383235de-84d7-43d4-9f3f-48c8a61ea5f3 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:59.237979 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-383235de-84d7-43d4-9f3f-48c8a61ea5f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '504cce2301a245d9935c44038a1d021e', 'request_id': 'req-383235de-84d7-43d4-9f3f-48c8a61ea5f3'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:59.244073 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-383235de-84d7-43d4-9f3f-48c8a61ea5f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '504cce2301a245d9935c44038a1d021e', 'request_id': 'req-383235de-84d7-43d4-9f3f-48c8a61ea5f3'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:59.250586 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-383235de-84d7-43d4-9f3f-48c8a61ea5f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '504cce2301a245d9935c44038a1d021e', 'request_id': 'req-383235de-84d7-43d4-9f3f-48c8a61ea5f3'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:59.263503 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-383235de-84d7-43d4-9f3f-48c8a61ea5f3 admin admin] The token cache is being invalidate because project 504cce2301a245d9935c44038a1d021e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:53:59.263798 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-383235de-84d7-43d4-9f3f-48c8a61ea5f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:59.264422 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-383235de-84d7-43d4-9f3f-48c8a61ea5f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:53:59.266012 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1266/2544] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:53:59 2022] DELETE /identity/v3/projects/504cce2301a245d9935c44038a1d021e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:53:59.271019 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97857b-9e5a-4335-a3a5-bf64229157d6 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:59.271316 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97857b-9e5a-4335-a3a5-bf64229157d6 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:59.271603 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97857b-9e5a-4335-a3a5-bf64229157d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:59.290559 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-eb97857b-9e5a-4335-a3a5-bf64229157d6 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:53:59.291695 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-eb97857b-9e5a-4335-a3a5-bf64229157d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:53:59.350598 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1279/2545] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:53:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:53:59.358282 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d564b-a7d2-454d-aa7e-f31b4a9bddbf None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:59.358580 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d564b-a7d2-454d-aa7e-f31b4a9bddbf None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:59.358986 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d564b-a7d2-454d-aa7e-f31b4a9bddbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:59.377886 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-4f7d564b-a7d2-454d-aa7e-f31b4a9bddbf None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:53:59.379259 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-4f7d564b-a7d2-454d-aa7e-f31b4a9bddbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:53:59.387805 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1267/2546] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:53:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:53:59.392000 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb97857b-9e5a-4335-a3a5-bf64229157d6 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:59.401689 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb97857b-9e5a-4335-a3a5-bf64229157d6 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:59.413281 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47f17d4-d4b6-4296-8651-7c282e7929a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:59.413955 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f17d4-d4b6-4296-8651-7c282e7929a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:59.414249 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f17d4-d4b6-4296-8651-7c282e7929a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:59.414558 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f17d4-d4b6-4296-8651-7c282e7929a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:59.415329 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47f17d4-d4b6-4296-8651-7c282e7929a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:53:59.415642 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47f17d4-d4b6-4296-8651-7c282e7929a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:59.417299 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47f17d4-d4b6-4296-8651-7c282e7929a9 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:59.429779 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1280/2547] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:53:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:53:59.434111 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d564b-a7d2-454d-aa7e-f31b4a9bddbf None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:59.443228 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d564b-a7d2-454d-aa7e-f31b4a9bddbf None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:59.448499 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc11cb0-7b37-4125-a223-06a97a5e5d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:59.449168 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc11cb0-7b37-4125-a223-06a97a5e5d63 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:59.449466 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc11cb0-7b37-4125-a223-06a97a5e5d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:59.449749 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc11cb0-7b37-4125-a223-06a97a5e5d63 admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:59.450435 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc11cb0-7b37-4125-a223-06a97a5e5d63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:59.452347 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc11cb0-7b37-4125-a223-06a97a5e5d63 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:59.479725 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1268/2548] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:53:59 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:53:59.483816 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47f17d4-d4b6-4296-8651-7c282e7929a9 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:59.499096 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47f17d4-d4b6-4296-8651-7c282e7929a9 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:59.527444 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:53:59.559540 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2066f1a-e1dc-4e0c-9183-ddabeb92bba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:59.560139 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2066f1a-e1dc-4e0c-9183-ddabeb92bba6 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:59.560445 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2066f1a-e1dc-4e0c-9183-ddabeb92bba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:59.560708 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2066f1a-e1dc-4e0c-9183-ddabeb92bba6 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:59.561440 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2066f1a-e1dc-4e0c-9183-ddabeb92bba6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:59.563386 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2066f1a-e1dc-4e0c-9183-ddabeb92bba6 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:59.584229 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1281/2549] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:53:59 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:53:59.586567 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc11cb0-7b37-4125-a223-06a97a5e5d63 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:59.596682 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc11cb0-7b37-4125-a223-06a97a5e5d63 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:59.601754 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da801537-01e8-42b8-a301-8ac7f7697f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:59.602429 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da801537-01e8-42b8-a301-8ac7f7697f5e admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:59.602699 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da801537-01e8-42b8-a301-8ac7f7697f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:59.602974 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da801537-01e8-42b8-a301-8ac7f7697f5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:59.603697 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da801537-01e8-42b8-a301-8ac7f7697f5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:59.605330 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da801537-01e8-42b8-a301-8ac7f7697f5e admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:59.616308 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1269/2550] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:53:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:53:59.620066 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2066f1a-e1dc-4e0c-9183-ddabeb92bba6 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:59.630225 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2066f1a-e1dc-4e0c-9183-ddabeb92bba6 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:59.635119 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e0af80-2f57-4f87-bc89-b3a14e9c308f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:59.635801 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0af80-2f57-4f87-bc89-b3a14e9c308f admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:59.636089 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0af80-2f57-4f87-bc89-b3a14e9c308f admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:59.636368 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0af80-2f57-4f87-bc89-b3a14e9c308f admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:59.637101 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e0af80-2f57-4f87-bc89-b3a14e9c308f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:59.638731 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e0af80-2f57-4f87-bc89-b3a14e9c308f admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:59.648601 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1282/2551] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:53:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:53:59.652441 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da801537-01e8-42b8-a301-8ac7f7697f5e admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:59.662806 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da801537-01e8-42b8-a301-8ac7f7697f5e admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:59.667262 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1257286-116e-4cbf-983c-76878e2edff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:59.668133 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1257286-116e-4cbf-983c-76878e2edff4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:59.668426 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1257286-116e-4cbf-983c-76878e2edff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:59.668706 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1257286-116e-4cbf-983c-76878e2edff4 admin admin] PATH_INFO: `/v3/projects/4ade3df8774942e3acdb22716b53e6d7/users/46f534186fab4dfd97460bbe682aadbd/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:59.683449 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1257286-116e-4cbf-983c-76878e2edff4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ade3df8774942e3acdb22716b53e6d7, user_id=46f534186fab4dfd97460bbe682aadbd, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:59.686118 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1257286-116e-4cbf-983c-76878e2edff4 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:59.698951 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1270/2552] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:53:59 2022] PUT /identity/v3/projects/4ade3df8774942e3acdb22716b53e6d7/users/46f534186fab4dfd97460bbe682aadbd/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:53:59.703919 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd421b64-f8d1-4b81-83c4-6dae0d3f86ff None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:59.704215 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd421b64-f8d1-4b81-83c4-6dae0d3f86ff None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:59.704497 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd421b64-f8d1-4b81-83c4-6dae0d3f86ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:59.722331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-bd421b64-f8d1-4b81-83c4-6dae0d3f86ff None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:53:59.723521 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-bd421b64-f8d1-4b81-83c4-6dae0d3f86ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:53:59.780556 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1283/2553] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:53:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:53:59.791190 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1257286-116e-4cbf-983c-76878e2edff4 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:53:59.792876 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-b1257286-116e-4cbf-983c-76878e2edff4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:53:59.799669 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:53:59.850699 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1257286-116e-4cbf-983c-76878e2edff4 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:53:59.854336 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3969721-0b1c-49f8-bbeb-b6bd52b496fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:53:59.854982 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3969721-0b1c-49f8-bbeb-b6bd52b496fb service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:53:59.855289 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3969721-0b1c-49f8-bbeb-b6bd52b496fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:53:59.855556 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3969721-0b1c-49f8-bbeb-b6bd52b496fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:53:59.864483 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3969721-0b1c-49f8-bbeb-b6bd52b496fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:53:59.866575 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3969721-0b1c-49f8-bbeb-b6bd52b496fb service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:53:59.876156 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1271/2554] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:53:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:54:00.481767 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c1ff93-6965-410f-9f6e-8861118751a1 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:00.482067 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c1ff93-6965-410f-9f6e-8861118751a1 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:00.482358 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c1ff93-6965-410f-9f6e-8861118751a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:00.509652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-43c1ff93-6965-410f-9f6e-8861118751a1 None None] MFA Rules not processed for user `46f534186fab4dfd97460bbe682aadbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:54:00.510878 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-43c1ff93-6965-410f-9f6e-8861118751a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:00.586162 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1284/2555] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:54:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:54:00.590602 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3969721-0b1c-49f8-bbeb-b6bd52b496fb service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:00.601186 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3969721-0b1c-49f8-bbeb-b6bd52b496fb service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:00.605758 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95597569-b0df-465b-942c-4464e4c9aead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:00.606412 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95597569-b0df-465b-942c-4464e4c9aead admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:00.606697 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95597569-b0df-465b-942c-4464e4c9aead admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:00.607030 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95597569-b0df-465b-942c-4464e4c9aead admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:00.607680 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95597569-b0df-465b-942c-4464e4c9aead admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:00.609622 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95597569-b0df-465b-942c-4464e4c9aead admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:00.641077 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1272/2556] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:54:00 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:54:00.642574 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c1ff93-6965-410f-9f6e-8861118751a1 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:00.653848 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c1ff93-6965-410f-9f6e-8861118751a1 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:00.687127 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:54:00.711693 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaef1bcf-790c-4572-8be9-d809ec3ea074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:00.712369 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef1bcf-790c-4572-8be9-d809ec3ea074 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:00.712656 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef1bcf-790c-4572-8be9-d809ec3ea074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:00.712936 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef1bcf-790c-4572-8be9-d809ec3ea074 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:00.713666 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaef1bcf-790c-4572-8be9-d809ec3ea074 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:00.715611 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaef1bcf-790c-4572-8be9-d809ec3ea074 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:00.734286 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1285/2557] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:54:00 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:54:00.738308 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95597569-b0df-465b-942c-4464e4c9aead admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:00.749275 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95597569-b0df-465b-942c-4464e4c9aead admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:00.753871 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e854ba-22e7-422b-85cc-b88e8ff47791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:00.754566 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e854ba-22e7-422b-85cc-b88e8ff47791 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:00.754857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e854ba-22e7-422b-85cc-b88e8ff47791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:00.755139 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e854ba-22e7-422b-85cc-b88e8ff47791 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:00.755876 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e854ba-22e7-422b-85cc-b88e8ff47791 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:00.758140 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e854ba-22e7-422b-85cc-b88e8ff47791 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:00.768451 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1273/2558] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:54:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:54:00.772354 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaef1bcf-790c-4572-8be9-d809ec3ea074 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:00.782249 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaef1bcf-790c-4572-8be9-d809ec3ea074 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:00.787733 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d47ce-2920-4e62-a7f9-45b3a479913e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:00.788602 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d47ce-2920-4e62-a7f9-45b3a479913e admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:00.788889 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d47ce-2920-4e62-a7f9-45b3a479913e admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:00.789193 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d47ce-2920-4e62-a7f9-45b3a479913e admin admin] PATH_INFO: `/v3/projects/ccb4633e5a3440caa6bcbb824d29b89b/users/f1a7e5fc6a3a45f6a4ffdc6bb3822263/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:00.804553 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7d47ce-2920-4e62-a7f9-45b3a479913e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccb4633e5a3440caa6bcbb824d29b89b, user_id=f1a7e5fc6a3a45f6a4ffdc6bb3822263, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:00.807347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7d47ce-2920-4e62-a7f9-45b3a479913e admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:00.820208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1286/2559] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:54:00 2022] PUT /identity/v3/projects/ccb4633e5a3440caa6bcbb824d29b89b/users/f1a7e5fc6a3a45f6a4ffdc6bb3822263/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:54:00.824300 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e854ba-22e7-422b-85cc-b88e8ff47791 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:00.833954 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e854ba-22e7-422b-85cc-b88e8ff47791 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:00.895660 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c35389-8f79-43b8-95f8-ed676ba310dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:00.896338 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c35389-8f79-43b8-95f8-ed676ba310dd admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:00.896672 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c35389-8f79-43b8-95f8-ed676ba310dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:00.896955 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c35389-8f79-43b8-95f8-ed676ba310dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:00.897749 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c35389-8f79-43b8-95f8-ed676ba310dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:00.899347 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c35389-8f79-43b8-95f8-ed676ba310dd admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:00.909019 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1274/2560] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:54:00 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:54:00.913138 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d47ce-2920-4e62-a7f9-45b3a479913e admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:00.923298 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d47ce-2920-4e62-a7f9-45b3a479913e admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:00.928582 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca96a8-8c3a-4e4c-9644-531c3ef6d779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:00.929494 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca96a8-8c3a-4e4c-9644-531c3ef6d779 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:00.929785 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca96a8-8c3a-4e4c-9644-531c3ef6d779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:00.930086 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca96a8-8c3a-4e4c-9644-531c3ef6d779 admin admin] PATH_INFO: `/v3/projects/ccb4633e5a3440caa6bcbb824d29b89b/users/f1a7e5fc6a3a45f6a4ffdc6bb3822263/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:00.932079 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ca96a8-8c3a-4e4c-9644-531c3ef6d779 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccb4633e5a3440caa6bcbb824d29b89b, user_id=f1a7e5fc6a3a45f6a4ffdc6bb3822263, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:00.934780 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ca96a8-8c3a-4e4c-9644-531c3ef6d779 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:00.946046 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1287/2561] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:54:00 2022] PUT /identity/v3/projects/ccb4633e5a3440caa6bcbb824d29b89b/users/f1a7e5fc6a3a45f6a4ffdc6bb3822263/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:54:01.575326 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425dcc91-f4c2-4e07-ba7d-9316dce1299d None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:01.575713 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425dcc91-f4c2-4e07-ba7d-9316dce1299d None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:01.576072 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425dcc91-f4c2-4e07-ba7d-9316dce1299d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:01.609256 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-425dcc91-f4c2-4e07-ba7d-9316dce1299d None None] MFA Rules not processed for user `f1a7e5fc6a3a45f6a4ffdc6bb3822263`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:54:01.610799 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-425dcc91-f4c2-4e07-ba7d-9316dce1299d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:01.673044 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:54:01.683271 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1275/2562] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:54:01 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:54:01.694760 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca96a8-8c3a-4e4c-9644-531c3ef6d779 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:01.697023 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b2ca96a8-8c3a-4e4c-9644-531c3ef6d779 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:01.753884 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca96a8-8c3a-4e4c-9644-531c3ef6d779 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:01.757563 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c9dc06-e944-4281-8902-056ca8ef61a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:01.758211 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9dc06-e944-4281-8902-056ca8ef61a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:01.758499 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9dc06-e944-4281-8902-056ca8ef61a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:01.758781 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9dc06-e944-4281-8902-056ca8ef61a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:01.815639 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c9dc06-e944-4281-8902-056ca8ef61a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:01.817113 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c9dc06-e944-4281-8902-056ca8ef61a7 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:01.826527 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1288/2563] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:54:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:54:03.164393 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425dcc91-f4c2-4e07-ba7d-9316dce1299d None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:03.165825 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-425dcc91-f4c2-4e07-ba7d-9316dce1299d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:03.177769 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425dcc91-f4c2-4e07-ba7d-9316dce1299d None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:03.181848 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1f1580-aa14-4c29-ad46-c6171a42d6fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:03.182490 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1f1580-aa14-4c29-ad46-c6171a42d6fb service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:03.182778 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1f1580-aa14-4c29-ad46-c6171a42d6fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:03.183056 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1f1580-aa14-4c29-ad46-c6171a42d6fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:03.191917 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1f1580-aa14-4c29-ad46-c6171a42d6fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:03.193408 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1f1580-aa14-4c29-ad46-c6171a42d6fb service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:03.202869 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1276/2564] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:54:03 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:54:03.713084 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c9dc06-e944-4281-8902-056ca8ef61a7 service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:03.721835 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c9dc06-e944-4281-8902-056ca8ef61a7 service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:03.725627 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b7d373-c167-4524-9151-3028e8adf077 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:03.726307 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7d373-c167-4524-9151-3028e8adf077 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:03.726599 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7d373-c167-4524-9151-3028e8adf077 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:03.726886 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7d373-c167-4524-9151-3028e8adf077 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:03.728714 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-12b7d373-c167-4524-9151-3028e8adf077 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:03.771787 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:54:03.811118 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b7d373-c167-4524-9151-3028e8adf077 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:03.812577 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b7d373-c167-4524-9151-3028e8adf077 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:03.831397 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1289/2565] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:54:03 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:54:06.913534 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1f1580-aa14-4c29-ad46-c6171a42d6fb service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:06.915315 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-9c1f1580-aa14-4c29-ad46-c6171a42d6fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:07.019242 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1f1580-aa14-4c29-ad46-c6171a42d6fb service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:07.025447 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e43df-319d-4aca-8211-e13d37a90f8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:07.026130 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e43df-319d-4aca-8211-e13d37a90f8b service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:07.026461 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e43df-319d-4aca-8211-e13d37a90f8b service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:07.026757 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e43df-319d-4aca-8211-e13d37a90f8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:07.028730 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-d67e43df-319d-4aca-8211-e13d37a90f8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:07.046691 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67e43df-319d-4aca-8211-e13d37a90f8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:07.048309 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67e43df-319d-4aca-8211-e13d37a90f8b service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:07.060404 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1277/2566] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:54:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:54:10.647699 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b7d373-c167-4524-9151-3028e8adf077 service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:10.706315 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b7d373-c167-4524-9151-3028e8adf077 service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:10.710549 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41fcc88-1458-4ba2-b44c-5bcbd7859b2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:10.711295 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41fcc88-1458-4ba2-b44c-5bcbd7859b2e service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:10.711583 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41fcc88-1458-4ba2-b44c-5bcbd7859b2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:10.711895 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41fcc88-1458-4ba2-b44c-5bcbd7859b2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:10.713806 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e41fcc88-1458-4ba2-b44c-5bcbd7859b2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:10.784603 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41fcc88-1458-4ba2-b44c-5bcbd7859b2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:10.786744 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41fcc88-1458-4ba2-b44c-5bcbd7859b2e service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:10.799362 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1290/2567] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:54:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:54:13.009641 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e43df-319d-4aca-8211-e13d37a90f8b service nova] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:13.011092 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-d67e43df-319d-4aca-8211-e13d37a90f8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:13.022369 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e43df-319d-4aca-8211-e13d37a90f8b service nova] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:13.026030 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1075e0c-c956-4c8b-a511-d0ffe2d5fe94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:13.026672 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1075e0c-c956-4c8b-a511-d0ffe2d5fe94 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:13.026960 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1075e0c-c956-4c8b-a511-d0ffe2d5fe94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:13.027242 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1075e0c-c956-4c8b-a511-d0ffe2d5fe94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:13.029139 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-f1075e0c-c956-4c8b-a511-d0ffe2d5fe94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:13.041129 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1075e0c-c956-4c8b-a511-d0ffe2d5fe94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:13.042965 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1075e0c-c956-4c8b-a511-d0ffe2d5fe94 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:13.051679 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1278/2568] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:54:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:54:14.144951 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41fcc88-1458-4ba2-b44c-5bcbd7859b2e service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:14.147146 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e41fcc88-1458-4ba2-b44c-5bcbd7859b2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:14.163420 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41fcc88-1458-4ba2-b44c-5bcbd7859b2e service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:14.169673 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47f62bc-dc80-400d-a005-247b8f221936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:14.171019 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f62bc-dc80-400d-a005-247b8f221936 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:14.171663 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f62bc-dc80-400d-a005-247b8f221936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:14.172149 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f62bc-dc80-400d-a005-247b8f221936 admin admin] PATH_INFO: `/v3/users/86b78ffb46894509bf8e81a154073885` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:14.174682 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47f62bc-dc80-400d-a005-247b8f221936 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86b78ffb46894509bf8e81a154073885)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:14.178362 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47f62bc-dc80-400d-a005-247b8f221936 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:14.226342 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-f47f62bc-dc80-400d-a005-247b8f221936 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86b78ffb46894509bf8e81a154073885'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:14.226993 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-f47f62bc-dc80-400d-a005-247b8f221936 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86b78ffb46894509bf8e81a154073885'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:14.227600 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-f47f62bc-dc80-400d-a005-247b8f221936 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86b78ffb46894509bf8e81a154073885'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:14.234170 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-f47f62bc-dc80-400d-a005-247b8f221936 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86b78ffb46894509bf8e81a154073885'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:14.238595 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:54:14.253215 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-f47f62bc-dc80-400d-a005-247b8f221936 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86b78ffb46894509bf8e81a154073885'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:14.271147 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1291/2569] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:54:14 2022] DELETE /identity/v3/users/86b78ffb46894509bf8e81a154073885 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:54:14.441632 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1075e0c-c956-4c8b-a511-d0ffe2d5fe94 service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:14.454190 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1075e0c-c956-4c8b-a511-d0ffe2d5fe94 service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:14.513662 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2a1633-56f9-4a0c-a50c-9156a2ae3ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:14.514527 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a1633-56f9-4a0c-a50c-9156a2ae3ab5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:14.514817 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a1633-56f9-4a0c-a50c-9156a2ae3ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:14.515100 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a1633-56f9-4a0c-a50c-9156a2ae3ab5 admin admin] PATH_INFO: `/v3/projects/bef5d1b475ef4228a77b87964b0b8ce1` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:14.516304 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2a1633-56f9-4a0c-a50c-9156a2ae3ab5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bef5d1b475ef4228a77b87964b0b8ce1)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:14.518260 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2a1633-56f9-4a0c-a50c-9156a2ae3ab5 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:14.574612 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-5a2a1633-56f9-4a0c-a50c-9156a2ae3ab5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bef5d1b475ef4228a77b87964b0b8ce1', 'request_id': 'req-5a2a1633-56f9-4a0c-a50c-9156a2ae3ab5'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:14.579201 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:54:14.580395 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-5a2a1633-56f9-4a0c-a50c-9156a2ae3ab5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bef5d1b475ef4228a77b87964b0b8ce1', 'request_id': 'req-5a2a1633-56f9-4a0c-a50c-9156a2ae3ab5'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:14.587995 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-5a2a1633-56f9-4a0c-a50c-9156a2ae3ab5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bef5d1b475ef4228a77b87964b0b8ce1', 'request_id': 'req-5a2a1633-56f9-4a0c-a50c-9156a2ae3ab5'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:14.599783 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-5a2a1633-56f9-4a0c-a50c-9156a2ae3ab5 admin admin] The token cache is being invalidate because project bef5d1b475ef4228a77b87964b0b8ce1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:54:14.600076 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-5a2a1633-56f9-4a0c-a50c-9156a2ae3ab5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:14.600709 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-5a2a1633-56f9-4a0c-a50c-9156a2ae3ab5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:14.603282 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1279/2570] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:54:14 2022] DELETE /identity/v3/projects/bef5d1b475ef4228a77b87964b0b8ce1 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:54:14.608281 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f96f93-16d0-4231-98d5-6e589ed73362 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:14.608582 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f96f93-16d0-4231-98d5-6e589ed73362 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:14.608866 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f96f93-16d0-4231-98d5-6e589ed73362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:14.626967 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-46f96f93-16d0-4231-98d5-6e589ed73362 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:54:14.628169 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-46f96f93-16d0-4231-98d5-6e589ed73362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:14.686438 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1292/2571] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:54:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:54:14.695515 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253053cd-e317-418f-8a3f-8e8d279b53f5 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:14.696001 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253053cd-e317-418f-8a3f-8e8d279b53f5 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:14.696289 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253053cd-e317-418f-8a3f-8e8d279b53f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:14.715476 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-253053cd-e317-418f-8a3f-8e8d279b53f5 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:54:14.716601 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-253053cd-e317-418f-8a3f-8e8d279b53f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:14.724987 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1280/2572] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:54:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:54:14.729116 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f96f93-16d0-4231-98d5-6e589ed73362 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:14.738752 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f96f93-16d0-4231-98d5-6e589ed73362 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:14.744085 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a31e684-bded-4643-bcd2-5e31aad95ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:14.744745 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31e684-bded-4643-bcd2-5e31aad95ac6 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:14.745035 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31e684-bded-4643-bcd2-5e31aad95ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:14.745347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31e684-bded-4643-bcd2-5e31aad95ac6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:14.746089 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a31e684-bded-4643-bcd2-5e31aad95ac6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:54:14.746407 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a31e684-bded-4643-bcd2-5e31aad95ac6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:14.748316 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a31e684-bded-4643-bcd2-5e31aad95ac6 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:14.760123 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1293/2573] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:54:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:54:14.764170 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253053cd-e317-418f-8a3f-8e8d279b53f5 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:14.773209 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253053cd-e317-418f-8a3f-8e8d279b53f5 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:14.777835 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f56e118-0ead-4b07-9872-e5f12819425b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:14.778486 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f56e118-0ead-4b07-9872-e5f12819425b admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:14.778777 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f56e118-0ead-4b07-9872-e5f12819425b admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:14.779059 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f56e118-0ead-4b07-9872-e5f12819425b admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:14.779746 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f56e118-0ead-4b07-9872-e5f12819425b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:14.781679 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f56e118-0ead-4b07-9872-e5f12819425b admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:14.809770 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1281/2574] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:54:14 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:54:14.813159 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a31e684-bded-4643-bcd2-5e31aad95ac6 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:14.823010 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a31e684-bded-4643-bcd2-5e31aad95ac6 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:14.887774 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237c462f-278f-4185-a3ee-ee1b796ff0fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:14.890822 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237c462f-278f-4185-a3ee-ee1b796ff0fc admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:14.891114 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237c462f-278f-4185-a3ee-ee1b796ff0fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:14.891404 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237c462f-278f-4185-a3ee-ee1b796ff0fc admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:14.892126 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237c462f-278f-4185-a3ee-ee1b796ff0fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:14.894199 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237c462f-278f-4185-a3ee-ee1b796ff0fc admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:14.921982 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1294/2575] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:54:14 2022] POST /identity/v3/users => generated 290 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:54:14.925906 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f56e118-0ead-4b07-9872-e5f12819425b admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:14.937969 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f56e118-0ead-4b07-9872-e5f12819425b admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:14.942604 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe802103-a4f2-40e5-bded-010b0bb37731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:14.943304 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe802103-a4f2-40e5-bded-010b0bb37731 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:14.943589 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe802103-a4f2-40e5-bded-010b0bb37731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:14.943867 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe802103-a4f2-40e5-bded-010b0bb37731 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:14.944597 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe802103-a4f2-40e5-bded-010b0bb37731 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:14.946268 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe802103-a4f2-40e5-bded-010b0bb37731 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:14.956960 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1282/2576] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:54:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:54:14.961461 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237c462f-278f-4185-a3ee-ee1b796ff0fc admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:14.972530 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237c462f-278f-4185-a3ee-ee1b796ff0fc admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:14.980474 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a98b0a-a034-49e2-af24-b92fe9085c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:14.981143 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a98b0a-a034-49e2-af24-b92fe9085c97 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:14.981460 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a98b0a-a034-49e2-af24-b92fe9085c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:14.981747 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a98b0a-a034-49e2-af24-b92fe9085c97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:14.982477 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a98b0a-a034-49e2-af24-b92fe9085c97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:14.984123 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a98b0a-a034-49e2-af24-b92fe9085c97 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:14.995140 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1295/2577] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:54:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:54:14.999937 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe802103-a4f2-40e5-bded-010b0bb37731 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:15.010212 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe802103-a4f2-40e5-bded-010b0bb37731 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:15.014838 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ad552-e15b-4552-8e46-1ee6f6bde532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:15.015721 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ad552-e15b-4552-8e46-1ee6f6bde532 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:15.016009 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ad552-e15b-4552-8e46-1ee6f6bde532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:15.016307 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ad552-e15b-4552-8e46-1ee6f6bde532 admin admin] PATH_INFO: `/v3/projects/1001ac5598f74a21b168777aa81d0581/users/2417170534314f98a0b549684f1a5f8b/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:15.034704 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5ad552-e15b-4552-8e46-1ee6f6bde532 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1001ac5598f74a21b168777aa81d0581, user_id=2417170534314f98a0b549684f1a5f8b, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:15.040126 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5ad552-e15b-4552-8e46-1ee6f6bde532 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:15.052525 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1283/2578] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:54:14 2022] PUT /identity/v3/projects/1001ac5598f74a21b168777aa81d0581/users/2417170534314f98a0b549684f1a5f8b/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:54:15.062604 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661d10eb-aea7-4834-9c30-8efb95045133 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:15.062899 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661d10eb-aea7-4834-9c30-8efb95045133 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:15.063182 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661d10eb-aea7-4834-9c30-8efb95045133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:15.089532 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-661d10eb-aea7-4834-9c30-8efb95045133 None None] MFA Rules not processed for user `2417170534314f98a0b549684f1a5f8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:54:15.090739 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-661d10eb-aea7-4834-9c30-8efb95045133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:15.120368 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:54:15.155658 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1296/2579] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:54:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:54:15.160184 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ad552-e15b-4552-8e46-1ee6f6bde532 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:15.220353 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ad552-e15b-4552-8e46-1ee6f6bde532 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:15.226267 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5818c1-149f-4992-9564-5641f3bf83cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:15.227153 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5818c1-149f-4992-9564-5641f3bf83cb admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:15.227531 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5818c1-149f-4992-9564-5641f3bf83cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:15.227855 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5818c1-149f-4992-9564-5641f3bf83cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:15.228633 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5818c1-149f-4992-9564-5641f3bf83cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:15.230724 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5818c1-149f-4992-9564-5641f3bf83cb admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:15.259091 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1284/2580] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:54:15 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:54:15.265013 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661d10eb-aea7-4834-9c30-8efb95045133 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:15.274975 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661d10eb-aea7-4834-9c30-8efb95045133 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:15.345795 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd73d4be-7123-4df2-9f38-a2607ab8d1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:15.346498 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd73d4be-7123-4df2-9f38-a2607ab8d1d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:15.346814 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd73d4be-7123-4df2-9f38-a2607ab8d1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:15.347098 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd73d4be-7123-4df2-9f38-a2607ab8d1d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:15.347808 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd73d4be-7123-4df2-9f38-a2607ab8d1d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:15.349850 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd73d4be-7123-4df2-9f38-a2607ab8d1d0 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:15.372496 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1297/2581] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:54:15 2022] POST /identity/v3/users => generated 297 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:54:15.373800 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5818c1-149f-4992-9564-5641f3bf83cb admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:15.384677 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5818c1-149f-4992-9564-5641f3bf83cb admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:15.389692 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec4998f-3424-4947-b23f-a246aaa5e9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:15.390363 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec4998f-3424-4947-b23f-a246aaa5e9af admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:15.390678 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec4998f-3424-4947-b23f-a246aaa5e9af admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:15.390958 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec4998f-3424-4947-b23f-a246aaa5e9af admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:15.391729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec4998f-3424-4947-b23f-a246aaa5e9af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:15.393334 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec4998f-3424-4947-b23f-a246aaa5e9af admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:15.404962 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1285/2582] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:54:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:54:15.409271 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd73d4be-7123-4df2-9f38-a2607ab8d1d0 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:15.419435 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd73d4be-7123-4df2-9f38-a2607ab8d1d0 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:15.424039 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76062b00-7d07-414b-87e6-b27d9634fe3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:15.424910 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76062b00-7d07-414b-87e6-b27d9634fe3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:15.425213 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76062b00-7d07-414b-87e6-b27d9634fe3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:15.425499 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76062b00-7d07-414b-87e6-b27d9634fe3f admin admin] PATH_INFO: `/v3/projects/9c9218576ed04d4ca015ffe347094c01/users/8eea2c4a52d54724b671ae10b7799682/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:15.440422 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76062b00-7d07-414b-87e6-b27d9634fe3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c9218576ed04d4ca015ffe347094c01, user_id=8eea2c4a52d54724b671ae10b7799682, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:15.443980 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76062b00-7d07-414b-87e6-b27d9634fe3f admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:15.460498 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1298/2583] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:54:15 2022] PUT /identity/v3/projects/9c9218576ed04d4ca015ffe347094c01/users/8eea2c4a52d54724b671ae10b7799682/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:54:15.464550 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec4998f-3424-4947-b23f-a246aaa5e9af admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:15.474685 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec4998f-3424-4947-b23f-a246aaa5e9af admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:15.527713 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:54:15.537034 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667c9481-ec36-4811-908d-980d8153fadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:15.538112 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c9481-ec36-4811-908d-980d8153fadc admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:15.538528 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c9481-ec36-4811-908d-980d8153fadc admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:15.539092 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c9481-ec36-4811-908d-980d8153fadc admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:15.539958 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667c9481-ec36-4811-908d-980d8153fadc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:15.541903 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667c9481-ec36-4811-908d-980d8153fadc admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:15.556331 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1286/2584] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:54:15 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:54:15.559283 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76062b00-7d07-414b-87e6-b27d9634fe3f admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:15.570412 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76062b00-7d07-414b-87e6-b27d9634fe3f admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:15.575218 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d5cc5-6f52-49b3-b327-cbf973d2c82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:15.576887 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d5cc5-6f52-49b3-b327-cbf973d2c82f admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:15.577295 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d5cc5-6f52-49b3-b327-cbf973d2c82f admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:15.577652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d5cc5-6f52-49b3-b327-cbf973d2c82f admin admin] PATH_INFO: `/v3/projects/9c9218576ed04d4ca015ffe347094c01/users/8eea2c4a52d54724b671ae10b7799682/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:15.579863 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79d5cc5-6f52-49b3-b327-cbf973d2c82f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c9218576ed04d4ca015ffe347094c01, user_id=8eea2c4a52d54724b671ae10b7799682, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:15.582697 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79d5cc5-6f52-49b3-b327-cbf973d2c82f admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:15.594118 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1299/2585] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:54:15 2022] PUT /identity/v3/projects/9c9218576ed04d4ca015ffe347094c01/users/8eea2c4a52d54724b671ae10b7799682/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:54:15.604245 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a13e58b-57b4-4154-ab7b-18c1920b8a34 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:15.604537 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a13e58b-57b4-4154-ab7b-18c1920b8a34 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:15.604885 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a13e58b-57b4-4154-ab7b-18c1920b8a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:15.631783 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-9a13e58b-57b4-4154-ab7b-18c1920b8a34 None None] MFA Rules not processed for user `8eea2c4a52d54724b671ae10b7799682`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:54:15.633112 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-9a13e58b-57b4-4154-ab7b-18c1920b8a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:15.702151 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1287/2586] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:54:15 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:54:15.714592 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d5cc5-6f52-49b3-b327-cbf973d2c82f admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:15.716415 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-c79d5cc5-6f52-49b3-b327-cbf973d2c82f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:15.775418 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d5cc5-6f52-49b3-b327-cbf973d2c82f admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:15.779617 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4422d250-150e-4509-ad06-c9e8404b6220 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:15.780282 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4422d250-150e-4509-ad06-c9e8404b6220 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:15.780700 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4422d250-150e-4509-ad06-c9e8404b6220 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:15.780994 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4422d250-150e-4509-ad06-c9e8404b6220 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:15.791517 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4422d250-150e-4509-ad06-c9e8404b6220 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:15.793031 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4422d250-150e-4509-ad06-c9e8404b6220 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:15.803174 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1300/2587] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:54:15 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:54:16.133993 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a13e58b-57b4-4154-ab7b-18c1920b8a34 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:16.143053 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a13e58b-57b4-4154-ab7b-18c1920b8a34 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:16.148262 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f5980d-af48-4176-a068-20e74386019e tempest-VolumeQuotaClassesTest-1579370969 tempest-VolumeQuotaClassesTest-1579370969-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea2c4a52d54724b671ae10b7799682', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c9218576ed04d4ca015ffe347094c01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:16.148925 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5980d-af48-4176-a068-20e74386019e tempest-VolumeQuotaClassesTest-1579370969 tempest-VolumeQuotaClassesTest-1579370969-project-admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:16.149255 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5980d-af48-4176-a068-20e74386019e tempest-VolumeQuotaClassesTest-1579370969 tempest-VolumeQuotaClassesTest-1579370969-project-admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:16.149529 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5980d-af48-4176-a068-20e74386019e tempest-VolumeQuotaClassesTest-1579370969 tempest-VolumeQuotaClassesTest-1579370969-project-admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:16.150261 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f5980d-af48-4176-a068-20e74386019e tempest-VolumeQuotaClassesTest-1579370969 tempest-VolumeQuotaClassesTest-1579370969-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:54:16.150577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f5980d-af48-4176-a068-20e74386019e tempest-VolumeQuotaClassesTest-1579370969 tempest-VolumeQuotaClassesTest-1579370969-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:16.152180 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f5980d-af48-4176-a068-20e74386019e tempest-VolumeQuotaClassesTest-1579370969 tempest-VolumeQuotaClassesTest-1579370969-project-admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:16.164083 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1288/2588] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:54:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:54:16.168317 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4422d250-150e-4509-ad06-c9e8404b6220 service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:16.178141 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4422d250-150e-4509-ad06-c9e8404b6220 service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:16.183610 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5083f66-4922-473e-9f43-095b4e956b0f tempest-VolumeQuotaClassesTest-1579370969 tempest-VolumeQuotaClassesTest-1579370969-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea2c4a52d54724b671ae10b7799682', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c9218576ed04d4ca015ffe347094c01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:16.184349 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5083f66-4922-473e-9f43-095b4e956b0f tempest-VolumeQuotaClassesTest-1579370969 tempest-VolumeQuotaClassesTest-1579370969-project-admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:16.184710 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5083f66-4922-473e-9f43-095b4e956b0f tempest-VolumeQuotaClassesTest-1579370969 tempest-VolumeQuotaClassesTest-1579370969-project-admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:16.185077 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5083f66-4922-473e-9f43-095b4e956b0f tempest-VolumeQuotaClassesTest-1579370969 tempest-VolumeQuotaClassesTest-1579370969-project-admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:16.185861 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5083f66-4922-473e-9f43-095b4e956b0f tempest-VolumeQuotaClassesTest-1579370969 tempest-VolumeQuotaClassesTest-1579370969-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:16.187867 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5083f66-4922-473e-9f43-095b4e956b0f tempest-VolumeQuotaClassesTest-1579370969 tempest-VolumeQuotaClassesTest-1579370969-project-admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:16.216056 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1301/2589] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:54:16 2022] POST /identity/v3/projects => generated 354 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:54:16.220262 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f5980d-af48-4176-a068-20e74386019e tempest-VolumeQuotaClassesTest-1579370969 tempest-VolumeQuotaClassesTest-1579370969-project-admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:16.230609 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f5980d-af48-4176-a068-20e74386019e tempest-VolumeQuotaClassesTest-1579370969 tempest-VolumeQuotaClassesTest-1579370969-project-admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:16.301887 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae777d2-43b1-47d9-86c6-b0d3f6b8d6f6 tempest-VolumeQuotaClassesTest-1579370969 tempest-VolumeQuotaClassesTest-1579370969-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea2c4a52d54724b671ae10b7799682', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c9218576ed04d4ca015ffe347094c01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:16.302690 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae777d2-43b1-47d9-86c6-b0d3f6b8d6f6 tempest-VolumeQuotaClassesTest-1579370969 tempest-VolumeQuotaClassesTest-1579370969-project-admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:16.303014 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae777d2-43b1-47d9-86c6-b0d3f6b8d6f6 tempest-VolumeQuotaClassesTest-1579370969 tempest-VolumeQuotaClassesTest-1579370969-project-admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:16.303451 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae777d2-43b1-47d9-86c6-b0d3f6b8d6f6 tempest-VolumeQuotaClassesTest-1579370969 tempest-VolumeQuotaClassesTest-1579370969-project-admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:16.304222 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae777d2-43b1-47d9-86c6-b0d3f6b8d6f6 tempest-VolumeQuotaClassesTest-1579370969 tempest-VolumeQuotaClassesTest-1579370969-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:54:16.304539 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae777d2-43b1-47d9-86c6-b0d3f6b8d6f6 tempest-VolumeQuotaClassesTest-1579370969 tempest-VolumeQuotaClassesTest-1579370969-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:16.306218 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae777d2-43b1-47d9-86c6-b0d3f6b8d6f6 tempest-VolumeQuotaClassesTest-1579370969 tempest-VolumeQuotaClassesTest-1579370969-project-admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:16.327889 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1289/2590] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:54:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:54:16.360493 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5083f66-4922-473e-9f43-095b4e956b0f tempest-VolumeQuotaClassesTest-1579370969 tempest-VolumeQuotaClassesTest-1579370969-project-admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:16.367404 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:54:16.372204 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5083f66-4922-473e-9f43-095b4e956b0f tempest-VolumeQuotaClassesTest-1579370969 tempest-VolumeQuotaClassesTest-1579370969-project-admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:16.378701 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e9db71-bcf0-4553-83cc-622181b4c4d1 tempest-VolumeQuotaClassesTest-1579370969 tempest-VolumeQuotaClassesTest-1579370969-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea2c4a52d54724b671ae10b7799682', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c9218576ed04d4ca015ffe347094c01', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:16.379636 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e9db71-bcf0-4553-83cc-622181b4c4d1 tempest-VolumeQuotaClassesTest-1579370969 tempest-VolumeQuotaClassesTest-1579370969-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:16.379929 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e9db71-bcf0-4553-83cc-622181b4c4d1 tempest-VolumeQuotaClassesTest-1579370969 tempest-VolumeQuotaClassesTest-1579370969-project-admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:16.380212 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e9db71-bcf0-4553-83cc-622181b4c4d1 tempest-VolumeQuotaClassesTest-1579370969 tempest-VolumeQuotaClassesTest-1579370969-project-admin] PATH_INFO: `/v3/projects/e35e0b41463940298601c426cfe2d3a6` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:16.382153 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e9db71-bcf0-4553-83cc-622181b4c4d1 tempest-VolumeQuotaClassesTest-1579370969 tempest-VolumeQuotaClassesTest-1579370969-project-admin] RBAC: Authorizing `identity:delete_project(project_id=e35e0b41463940298601c426cfe2d3a6)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:16.384298 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e9db71-bcf0-4553-83cc-622181b4c4d1 tempest-VolumeQuotaClassesTest-1579370969 tempest-VolumeQuotaClassesTest-1579370969-project-admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:16.400585 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae777d2-43b1-47d9-86c6-b0d3f6b8d6f6 tempest-VolumeQuotaClassesTest-1579370969 tempest-VolumeQuotaClassesTest-1579370969-project-admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:16.402431 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-0ae777d2-43b1-47d9-86c6-b0d3f6b8d6f6 tempest-VolumeQuotaClassesTest-1579370969 tempest-VolumeQuotaClassesTest-1579370969-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:16.438317 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a5e9db71-bcf0-4553-83cc-622181b4c4d1 tempest-VolumeQuotaClassesTest-1579370969 tempest-VolumeQuotaClassesTest-1579370969-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e35e0b41463940298601c426cfe2d3a6', 'request_id': 'req-a5e9db71-bcf0-4553-83cc-622181b4c4d1'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:16.444773 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a5e9db71-bcf0-4553-83cc-622181b4c4d1 tempest-VolumeQuotaClassesTest-1579370969 tempest-VolumeQuotaClassesTest-1579370969-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e35e0b41463940298601c426cfe2d3a6', 'request_id': 'req-a5e9db71-bcf0-4553-83cc-622181b4c4d1'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:16.451209 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a5e9db71-bcf0-4553-83cc-622181b4c4d1 tempest-VolumeQuotaClassesTest-1579370969 tempest-VolumeQuotaClassesTest-1579370969-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e35e0b41463940298601c426cfe2d3a6', 'request_id': 'req-a5e9db71-bcf0-4553-83cc-622181b4c4d1'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:16.463748 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a5e9db71-bcf0-4553-83cc-622181b4c4d1 tempest-VolumeQuotaClassesTest-1579370969 tempest-VolumeQuotaClassesTest-1579370969-project-admin] The token cache is being invalidate because project e35e0b41463940298601c426cfe2d3a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:54:16.464025 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a5e9db71-bcf0-4553-83cc-622181b4c4d1 tempest-VolumeQuotaClassesTest-1579370969 tempest-VolumeQuotaClassesTest-1579370969-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:16.464678 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a5e9db71-bcf0-4553-83cc-622181b4c4d1 tempest-VolumeQuotaClassesTest-1579370969 tempest-VolumeQuotaClassesTest-1579370969-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:16.466313 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1302/2591] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:54:16 2022] DELETE /identity/v3/projects/e35e0b41463940298601c426cfe2d3a6 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:54:16.470210 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:54:16.473525 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae777d2-43b1-47d9-86c6-b0d3f6b8d6f6 tempest-VolumeQuotaClassesTest-1579370969 tempest-VolumeQuotaClassesTest-1579370969-project-admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:16.535929 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65259fa5-baef-4c2a-9378-5b7d76ab6f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:16.536735 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65259fa5-baef-4c2a-9378-5b7d76ab6f40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:16.537014 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65259fa5-baef-4c2a-9378-5b7d76ab6f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:16.537332 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65259fa5-baef-4c2a-9378-5b7d76ab6f40 admin admin] PATH_INFO: `/v3/users/46f534186fab4dfd97460bbe682aadbd` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:16.538581 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65259fa5-baef-4c2a-9378-5b7d76ab6f40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46f534186fab4dfd97460bbe682aadbd)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:16.540550 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65259fa5-baef-4c2a-9378-5b7d76ab6f40 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:16.594129 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-65259fa5-baef-4c2a-9378-5b7d76ab6f40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46f534186fab4dfd97460bbe682aadbd'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:16.594129 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-65259fa5-baef-4c2a-9378-5b7d76ab6f40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46f534186fab4dfd97460bbe682aadbd'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:16.600978 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-65259fa5-baef-4c2a-9378-5b7d76ab6f40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46f534186fab4dfd97460bbe682aadbd'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:16.617124 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-65259fa5-baef-4c2a-9378-5b7d76ab6f40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46f534186fab4dfd97460bbe682aadbd'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:16.617768 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-65259fa5-baef-4c2a-9378-5b7d76ab6f40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46f534186fab4dfd97460bbe682aadbd'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:16.637742 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1290/2592] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:54:16 2022] DELETE /identity/v3/users/46f534186fab4dfd97460bbe682aadbd => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:54:16.674580 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e9db71-bcf0-4553-83cc-622181b4c4d1 tempest-VolumeQuotaClassesTest-1579370969 tempest-VolumeQuotaClassesTest-1579370969-project-admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:16.676041 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-a5e9db71-bcf0-4553-83cc-622181b4c4d1 tempest-VolumeQuotaClassesTest-1579370969 tempest-VolumeQuotaClassesTest-1579370969-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:16.689952 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e9db71-bcf0-4553-83cc-622181b4c4d1 tempest-VolumeQuotaClassesTest-1579370969 tempest-VolumeQuotaClassesTest-1579370969-project-admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:16.747863 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c43e2-6cd4-481d-8a9f-4e399b0961a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:16.748867 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c43e2-6cd4-481d-8a9f-4e399b0961a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:16.749175 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c43e2-6cd4-481d-8a9f-4e399b0961a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:16.749479 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c43e2-6cd4-481d-8a9f-4e399b0961a7 admin admin] PATH_INFO: `/v3/users/2417170534314f98a0b549684f1a5f8b` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:16.750722 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8c43e2-6cd4-481d-8a9f-4e399b0961a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2417170534314f98a0b549684f1a5f8b)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:16.752671 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8c43e2-6cd4-481d-8a9f-4e399b0961a7 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:16.805119 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-2b8c43e2-6cd4-481d-8a9f-4e399b0961a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2417170534314f98a0b549684f1a5f8b'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:16.805856 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-2b8c43e2-6cd4-481d-8a9f-4e399b0961a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2417170534314f98a0b549684f1a5f8b'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:16.806473 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-2b8c43e2-6cd4-481d-8a9f-4e399b0961a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2417170534314f98a0b549684f1a5f8b'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:16.814667 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-2b8c43e2-6cd4-481d-8a9f-4e399b0961a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2417170534314f98a0b549684f1a5f8b'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:16.821181 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65259fa5-baef-4c2a-9378-5b7d76ab6f40 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:16.823276 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-65259fa5-baef-4c2a-9378-5b7d76ab6f40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:16.833828 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-2b8c43e2-6cd4-481d-8a9f-4e399b0961a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2417170534314f98a0b549684f1a5f8b'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:16.847704 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1303/2593] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:54:16 2022] DELETE /identity/v3/users/2417170534314f98a0b549684f1a5f8b => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:54:16.852515 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a053dd92-36e7-4530-83a0-9f4e68323232 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:16.852804 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a053dd92-36e7-4530-83a0-9f4e68323232 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:16.853129 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a053dd92-36e7-4530-83a0-9f4e68323232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:16.855173 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65259fa5-baef-4c2a-9378-5b7d76ab6f40 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:16.864619 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:54:16.872241 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-a053dd92-36e7-4530-83a0-9f4e68323232 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:54:16.874130 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-a053dd92-36e7-4530-83a0-9f4e68323232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:16.917654 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2377f1f-278b-4aaa-ba21-849465d3bb7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:16.918528 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2377f1f-278b-4aaa-ba21-849465d3bb7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:16.918824 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2377f1f-278b-4aaa-ba21-849465d3bb7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:16.919111 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2377f1f-278b-4aaa-ba21-849465d3bb7a admin admin] PATH_INFO: `/v3/projects/4ade3df8774942e3acdb22716b53e6d7` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:16.920320 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2377f1f-278b-4aaa-ba21-849465d3bb7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ade3df8774942e3acdb22716b53e6d7)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:16.926146 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2377f1f-278b-4aaa-ba21-849465d3bb7a admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:16.944612 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1304/2594] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:54:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:54:16.954359 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a053dd92-36e7-4530-83a0-9f4e68323232 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:16.955739 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-a053dd92-36e7-4530-83a0-9f4e68323232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:16.958603 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:54:16.980159 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-a2377f1f-278b-4aaa-ba21-849465d3bb7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ade3df8774942e3acdb22716b53e6d7', 'request_id': 'req-a2377f1f-278b-4aaa-ba21-849465d3bb7a'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:16.986771 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-a2377f1f-278b-4aaa-ba21-849465d3bb7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ade3df8774942e3acdb22716b53e6d7', 'request_id': 'req-a2377f1f-278b-4aaa-ba21-849465d3bb7a'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:16.993608 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-a2377f1f-278b-4aaa-ba21-849465d3bb7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ade3df8774942e3acdb22716b53e6d7', 'request_id': 'req-a2377f1f-278b-4aaa-ba21-849465d3bb7a'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:17.007251 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-a2377f1f-278b-4aaa-ba21-849465d3bb7a admin admin] The token cache is being invalidate because project 4ade3df8774942e3acdb22716b53e6d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:54:17.007543 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-a2377f1f-278b-4aaa-ba21-849465d3bb7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:17.008172 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-a2377f1f-278b-4aaa-ba21-849465d3bb7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:17.009738 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1291/2595] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:54:16 2022] DELETE /identity/v3/projects/4ade3df8774942e3acdb22716b53e6d7 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:54:17.013435 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2377f1f-278b-4aaa-ba21-849465d3bb7a admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:17.014711 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-a2377f1f-278b-4aaa-ba21-849465d3bb7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:17.018417 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a053dd92-36e7-4530-83a0-9f4e68323232 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:17.069204 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b10b96-c20e-4673-b134-ab9452330b66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:17.069783 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b10b96-c20e-4673-b134-ab9452330b66 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:17.070076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b10b96-c20e-4673-b134-ab9452330b66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:17.070364 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b10b96-c20e-4673-b134-ab9452330b66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:17.072137 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-77b10b96-c20e-4673-b134-ab9452330b66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:17.078270 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2377f1f-278b-4aaa-ba21-849465d3bb7a admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:17.082851 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46b6315-3ae3-462c-a8b1-9abe3d0802f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:17.083732 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b6315-3ae3-462c-a8b1-9abe3d0802f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:17.083967 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b10b96-c20e-4673-b134-ab9452330b66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:17.084085 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b6315-3ae3-462c-a8b1-9abe3d0802f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:17.084273 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b6315-3ae3-462c-a8b1-9abe3d0802f6 admin admin] PATH_INFO: `/v3/users/f1a7e5fc6a3a45f6a4ffdc6bb3822263` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:17.085448 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b10b96-c20e-4673-b134-ab9452330b66 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:17.085566 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46b6315-3ae3-462c-a8b1-9abe3d0802f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1a7e5fc6a3a45f6a4ffdc6bb3822263)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:17.087385 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46b6315-3ae3-462c-a8b1-9abe3d0802f6 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:17.094863 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1305/2596] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:54:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 16:54:17.137273 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-d46b6315-3ae3-462c-a8b1-9abe3d0802f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1a7e5fc6a3a45f6a4ffdc6bb3822263'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:17.137955 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-d46b6315-3ae3-462c-a8b1-9abe3d0802f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1a7e5fc6a3a45f6a4ffdc6bb3822263'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:17.144973 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-d46b6315-3ae3-462c-a8b1-9abe3d0802f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1a7e5fc6a3a45f6a4ffdc6bb3822263'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:17.158001 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-d46b6315-3ae3-462c-a8b1-9abe3d0802f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1a7e5fc6a3a45f6a4ffdc6bb3822263'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:17.158700 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-d46b6315-3ae3-462c-a8b1-9abe3d0802f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1a7e5fc6a3a45f6a4ffdc6bb3822263'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:17.172341 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1292/2597] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:54:17 2022] DELETE /identity/v3/users/f1a7e5fc6a3a45f6a4ffdc6bb3822263 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:54:17.346832 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b10b96-c20e-4673-b134-ab9452330b66 service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:17.360403 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b10b96-c20e-4673-b134-ab9452330b66 service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:17.391391 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46b6315-3ae3-462c-a8b1-9abe3d0802f6 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:17.393012 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-d46b6315-3ae3-462c-a8b1-9abe3d0802f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:17.435515 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:54:17.436526 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbe4173-b219-406f-9291-f2d60126a1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:17.437923 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe4173-b219-406f-9291-f2d60126a1a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:17.438219 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe4173-b219-406f-9291-f2d60126a1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:17.438505 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe4173-b219-406f-9291-f2d60126a1a6 admin admin] PATH_INFO: `/v3/projects/ccb4633e5a3440caa6bcbb824d29b89b` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:17.439726 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbe4173-b219-406f-9291-f2d60126a1a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccb4633e5a3440caa6bcbb824d29b89b)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:17.441843 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbe4173-b219-406f-9291-f2d60126a1a6 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:17.451097 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=100 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:54:17.470383 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46b6315-3ae3-462c-a8b1-9abe3d0802f6 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:17.477267 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d6c45-930f-4bfd-9f76-eeec184c3337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:17.478366 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d6c45-930f-4bfd-9f76-eeec184c3337 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:17.478665 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d6c45-930f-4bfd-9f76-eeec184c3337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:17.478957 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d6c45-930f-4bfd-9f76-eeec184c3337 admin admin] PATH_INFO: `/v3/projects/1001ac5598f74a21b168777aa81d0581` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:17.480300 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185d6c45-930f-4bfd-9f76-eeec184c3337 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1001ac5598f74a21b168777aa81d0581)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:17.482338 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185d6c45-930f-4bfd-9f76-eeec184c3337 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:17.497663 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-adbe4173-b219-406f-9291-f2d60126a1a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccb4633e5a3440caa6bcbb824d29b89b', 'request_id': 'req-adbe4173-b219-406f-9291-f2d60126a1a6'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:17.503307 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-adbe4173-b219-406f-9291-f2d60126a1a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccb4633e5a3440caa6bcbb824d29b89b', 'request_id': 'req-adbe4173-b219-406f-9291-f2d60126a1a6'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:17.509656 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-adbe4173-b219-406f-9291-f2d60126a1a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccb4633e5a3440caa6bcbb824d29b89b', 'request_id': 'req-adbe4173-b219-406f-9291-f2d60126a1a6'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:17.523263 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-adbe4173-b219-406f-9291-f2d60126a1a6 admin admin] The token cache is being invalidate because project ccb4633e5a3440caa6bcbb824d29b89b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:54:17.523639 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-adbe4173-b219-406f-9291-f2d60126a1a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:17.524351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-adbe4173-b219-406f-9291-f2d60126a1a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:17.526054 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1306/2598] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:54:17 2022] DELETE /identity/v3/projects/ccb4633e5a3440caa6bcbb824d29b89b => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:54:17.531630 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1ecbb-2e2d-4e73-a4a4-f2dfa00e1e12 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:17.532001 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1ecbb-2e2d-4e73-a4a4-f2dfa00e1e12 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:17.532498 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1ecbb-2e2d-4e73-a4a4-f2dfa00e1e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:17.533799 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-185d6c45-930f-4bfd-9f76-eeec184c3337 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1001ac5598f74a21b168777aa81d0581', 'request_id': 'req-185d6c45-930f-4bfd-9f76-eeec184c3337'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:17.540492 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-185d6c45-930f-4bfd-9f76-eeec184c3337 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1001ac5598f74a21b168777aa81d0581', 'request_id': 'req-185d6c45-930f-4bfd-9f76-eeec184c3337'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:17.546890 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-185d6c45-930f-4bfd-9f76-eeec184c3337 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1001ac5598f74a21b168777aa81d0581', 'request_id': 'req-185d6c45-930f-4bfd-9f76-eeec184c3337'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:17.551560 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-74f1ecbb-2e2d-4e73-a4a4-f2dfa00e1e12 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:54:17.553085 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-74f1ecbb-2e2d-4e73-a4a4-f2dfa00e1e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:17.560079 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-185d6c45-930f-4bfd-9f76-eeec184c3337 admin admin] The token cache is being invalidate because project 1001ac5598f74a21b168777aa81d0581 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:54:17.560455 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-185d6c45-930f-4bfd-9f76-eeec184c3337 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:17.561143 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-185d6c45-930f-4bfd-9f76-eeec184c3337 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:17.562793 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1293/2599] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:54:17 2022] DELETE /identity/v3/projects/1001ac5598f74a21b168777aa81d0581 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:54:17.566794 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d6c45-930f-4bfd-9f76-eeec184c3337 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:17.568746 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-185d6c45-930f-4bfd-9f76-eeec184c3337 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:17.617545 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1307/2600] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:54:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:54:17.625677 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b042fb-ea81-4559-93e0-e80758757683 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:17.625977 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b042fb-ea81-4559-93e0-e80758757683 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:17.626392 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b042fb-ea81-4559-93e0-e80758757683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:17.636111 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d6c45-930f-4bfd-9f76-eeec184c3337 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:17.640796 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a8182-ae8d-4612-8c1e-7d46517c82a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:17.641715 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a8182-ae8d-4612-8c1e-7d46517c82a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:17.642004 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a8182-ae8d-4612-8c1e-7d46517c82a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:17.642281 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a8182-ae8d-4612-8c1e-7d46517c82a1 admin admin] PATH_INFO: `/v3/users/8eea2c4a52d54724b671ae10b7799682` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:17.643490 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6a8182-ae8d-4612-8c1e-7d46517c82a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8eea2c4a52d54724b671ae10b7799682)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:17.644841 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-80b042fb-ea81-4559-93e0-e80758757683 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:54:17.645447 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6a8182-ae8d-4612-8c1e-7d46517c82a1 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:17.646084 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-80b042fb-ea81-4559-93e0-e80758757683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:17.654551 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1308/2601] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:54:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:54:17.658667 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b042fb-ea81-4559-93e0-e80758757683 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:17.667824 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b042fb-ea81-4559-93e0-e80758757683 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:17.672698 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4271372-3d7a-4b1c-b3fc-3561f614704b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:17.673386 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4271372-3d7a-4b1c-b3fc-3561f614704b admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:17.673680 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4271372-3d7a-4b1c-b3fc-3561f614704b admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:17.673964 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4271372-3d7a-4b1c-b3fc-3561f614704b admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:17.674788 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4271372-3d7a-4b1c-b3fc-3561f614704b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:54:17.675110 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4271372-3d7a-4b1c-b3fc-3561f614704b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:17.676764 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4271372-3d7a-4b1c-b3fc-3561f614704b admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:17.688824 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1309/2602] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:54:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:54:17.692933 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4271372-3d7a-4b1c-b3fc-3561f614704b admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:17.695910 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0c6a8182-ae8d-4612-8c1e-7d46517c82a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8eea2c4a52d54724b671ae10b7799682'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:17.696616 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0c6a8182-ae8d-4612-8c1e-7d46517c82a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8eea2c4a52d54724b671ae10b7799682'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:17.703230 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4271372-3d7a-4b1c-b3fc-3561f614704b admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:17.704654 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0c6a8182-ae8d-4612-8c1e-7d46517c82a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8eea2c4a52d54724b671ae10b7799682'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:17.708105 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793f35de-02ed-4cbb-962b-61137f4e252d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:17.709352 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f35de-02ed-4cbb-962b-61137f4e252d admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:17.709646 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f35de-02ed-4cbb-962b-61137f4e252d admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:17.709932 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f35de-02ed-4cbb-962b-61137f4e252d admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:17.710637 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793f35de-02ed-4cbb-962b-61137f4e252d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:17.712538 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793f35de-02ed-4cbb-962b-61137f4e252d admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:17.713947 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:54:17.719283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0c6a8182-ae8d-4612-8c1e-7d46517c82a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8eea2c4a52d54724b671ae10b7799682'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:17.720090 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0c6a8182-ae8d-4612-8c1e-7d46517c82a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8eea2c4a52d54724b671ae10b7799682'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:17.735842 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1294/2603] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:54:17 2022] DELETE /identity/v3/users/8eea2c4a52d54724b671ae10b7799682 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:54:17.739809 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1310/2604] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:54:17 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:54:17.743740 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a8182-ae8d-4612-8c1e-7d46517c82a1 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:17.745121 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-0c6a8182-ae8d-4612-8c1e-7d46517c82a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:17.813252 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a8182-ae8d-4612-8c1e-7d46517c82a1 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:17.873659 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb47ec-606c-4dfd-957d-0c632d2f9e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:17.874337 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb47ec-606c-4dfd-957d-0c632d2f9e64 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:17.874624 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb47ec-606c-4dfd-957d-0c632d2f9e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:17.874906 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb47ec-606c-4dfd-957d-0c632d2f9e64 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:17.875636 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bb47ec-606c-4dfd-957d-0c632d2f9e64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:17.877598 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bb47ec-606c-4dfd-957d-0c632d2f9e64 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:17.904158 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1295/2605] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:54:17 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:54:17.908282 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793f35de-02ed-4cbb-962b-61137f4e252d admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:17.918770 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793f35de-02ed-4cbb-962b-61137f4e252d admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:17.926593 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d02d3-0bf3-4c0d-bc91-2ec9413db8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:17.926593 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d02d3-0bf3-4c0d-bc91-2ec9413db8f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:17.926593 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d02d3-0bf3-4c0d-bc91-2ec9413db8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:17.926593 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d02d3-0bf3-4c0d-bc91-2ec9413db8f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:17.927034 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811d02d3-0bf3-4c0d-bc91-2ec9413db8f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:17.928557 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811d02d3-0bf3-4c0d-bc91-2ec9413db8f1 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:17.939398 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1311/2606] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:54:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:54:17.944237 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb47ec-606c-4dfd-957d-0c632d2f9e64 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:17.956019 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb47ec-606c-4dfd-957d-0c632d2f9e64 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:17.963684 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf9313f-0f63-4ecf-8a0c-66973c8a2b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:17.964210 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9313f-0f63-4ecf-8a0c-66973c8a2b22 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:17.964497 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9313f-0f63-4ecf-8a0c-66973c8a2b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:17.964776 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9313f-0f63-4ecf-8a0c-66973c8a2b22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:17.965546 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf9313f-0f63-4ecf-8a0c-66973c8a2b22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:17.967152 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf9313f-0f63-4ecf-8a0c-66973c8a2b22 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:17.978503 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1296/2607] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:54:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:54:17.982724 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d02d3-0bf3-4c0d-bc91-2ec9413db8f1 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:17.993508 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d02d3-0bf3-4c0d-bc91-2ec9413db8f1 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:17.998338 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f61a06-7a86-47fd-9129-82bd2a2b4d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:17.999216 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f61a06-7a86-47fd-9129-82bd2a2b4d1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:17.999617 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f61a06-7a86-47fd-9129-82bd2a2b4d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:18.000259 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f61a06-7a86-47fd-9129-82bd2a2b4d1a admin admin] PATH_INFO: `/v3/projects/7511afdd52d341f9a1dad3b628654693/users/367eca85f09544aaa20741d1bfa7ff14/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:18.015372 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f61a06-7a86-47fd-9129-82bd2a2b4d1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7511afdd52d341f9a1dad3b628654693, user_id=367eca85f09544aaa20741d1bfa7ff14, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:18.018899 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f61a06-7a86-47fd-9129-82bd2a2b4d1a admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:18.027578 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf9313f-0f63-4ecf-8a0c-66973c8a2b22 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:18.029971 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ebf9313f-0f63-4ecf-8a0c-66973c8a2b22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:18.041617 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1312/2608] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:54:17 2022] PUT /identity/v3/projects/7511afdd52d341f9a1dad3b628654693/users/367eca85f09544aaa20741d1bfa7ff14/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:54:18.045186 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf9313f-0f63-4ecf-8a0c-66973c8a2b22 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:18.051611 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879dd7a8-b194-4f4c-8e08-6c913a5c0a2e None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:18.051982 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879dd7a8-b194-4f4c-8e08-6c913a5c0a2e None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:18.052350 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879dd7a8-b194-4f4c-8e08-6c913a5c0a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:18.079600 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-879dd7a8-b194-4f4c-8e08-6c913a5c0a2e None None] MFA Rules not processed for user `367eca85f09544aaa20741d1bfa7ff14`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:54:18.080870 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-879dd7a8-b194-4f4c-8e08-6c913a5c0a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:18.089117 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:54:18.108676 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7513c01f-4fac-42fc-a299-6a5c664c4545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:18.109643 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7513c01f-4fac-42fc-a299-6a5c664c4545 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:18.109933 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7513c01f-4fac-42fc-a299-6a5c664c4545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:18.110218 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7513c01f-4fac-42fc-a299-6a5c664c4545 admin admin] PATH_INFO: `/v3/projects/9c9218576ed04d4ca015ffe347094c01` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:18.111432 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7513c01f-4fac-42fc-a299-6a5c664c4545 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c9218576ed04d4ca015ffe347094c01)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:18.113391 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7513c01f-4fac-42fc-a299-6a5c664c4545 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:18.126184 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:54:18.147244 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1313/2609] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:54:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:54:18.158094 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879dd7a8-b194-4f4c-8e08-6c913a5c0a2e None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:18.159430 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-879dd7a8-b194-4f4c-8e08-6c913a5c0a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:18.164385 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-7513c01f-4fac-42fc-a299-6a5c664c4545 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c9218576ed04d4ca015ffe347094c01', 'request_id': 'req-7513c01f-4fac-42fc-a299-6a5c664c4545'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:18.170649 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-7513c01f-4fac-42fc-a299-6a5c664c4545 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c9218576ed04d4ca015ffe347094c01', 'request_id': 'req-7513c01f-4fac-42fc-a299-6a5c664c4545'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:18.179276 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-7513c01f-4fac-42fc-a299-6a5c664c4545 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c9218576ed04d4ca015ffe347094c01', 'request_id': 'req-7513c01f-4fac-42fc-a299-6a5c664c4545'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:18.191213 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-7513c01f-4fac-42fc-a299-6a5c664c4545 admin admin] The token cache is being invalidate because project 9c9218576ed04d4ca015ffe347094c01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:54:18.191504 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-7513c01f-4fac-42fc-a299-6a5c664c4545 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:18.192142 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-7513c01f-4fac-42fc-a299-6a5c664c4545 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:18.194527 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1297/2610] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:54:18 2022] DELETE /identity/v3/projects/9c9218576ed04d4ca015ffe347094c01 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:54:18.198866 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f812146-dfa2-4236-a041-b54e39fe538a None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:18.199161 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f812146-dfa2-4236-a041-b54e39fe538a None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:18.199442 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f812146-dfa2-4236-a041-b54e39fe538a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:18.216671 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-0f812146-dfa2-4236-a041-b54e39fe538a None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:54:18.217931 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-0f812146-dfa2-4236-a041-b54e39fe538a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:18.218365 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879dd7a8-b194-4f4c-8e08-6c913a5c0a2e None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:18.276873 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39462f8d-204e-4f26-9804-b80143339c4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:18.277560 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39462f8d-204e-4f26-9804-b80143339c4a service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:18.277852 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39462f8d-204e-4f26-9804-b80143339c4a service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:18.278135 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39462f8d-204e-4f26-9804-b80143339c4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:18.279985 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-39462f8d-204e-4f26-9804-b80143339c4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:18.284870 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1298/2611] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:54:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:54:18.294341 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a296b8-64f0-4610-ae77-3ac360511f94 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:18.294634 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a296b8-64f0-4610-ae77-3ac360511f94 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:18.294918 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a296b8-64f0-4610-ae77-3ac360511f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:18.313388 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-03a296b8-64f0-4610-ae77-3ac360511f94 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:54:18.314594 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-03a296b8-64f0-4610-ae77-3ac360511f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:18.322535 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1299/2612] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:54:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:54:18.326543 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a296b8-64f0-4610-ae77-3ac360511f94 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:18.335256 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a296b8-64f0-4610-ae77-3ac360511f94 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:18.338676 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39462f8d-204e-4f26-9804-b80143339c4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:18.340152 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39462f8d-204e-4f26-9804-b80143339c4a service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:18.340415 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b2a45c-ff1a-47a6-9ff0-96db4a970701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:18.341060 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b2a45c-ff1a-47a6-9ff0-96db4a970701 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:18.341387 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b2a45c-ff1a-47a6-9ff0-96db4a970701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:18.341673 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b2a45c-ff1a-47a6-9ff0-96db4a970701 admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:18.342407 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b2a45c-ff1a-47a6-9ff0-96db4a970701 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:54:18.342721 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b2a45c-ff1a-47a6-9ff0-96db4a970701 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:18.344320 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b2a45c-ff1a-47a6-9ff0-96db4a970701 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:18.349806 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1314/2613] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:54:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 16:54:18.357182 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1300/2614] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:54:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:54:18.360676 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39462f8d-204e-4f26-9804-b80143339c4a service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:18.372152 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39462f8d-204e-4f26-9804-b80143339c4a service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:18.376752 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f69fd1-adcd-4112-92ce-56973e84ec35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:18.377376 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f69fd1-adcd-4112-92ce-56973e84ec35 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:18.377661 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f69fd1-adcd-4112-92ce-56973e84ec35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:18.377937 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f69fd1-adcd-4112-92ce-56973e84ec35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:18.378632 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f69fd1-adcd-4112-92ce-56973e84ec35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:18.380552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f69fd1-adcd-4112-92ce-56973e84ec35 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:18.399868 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:54:18.408702 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1315/2615] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:54:18 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:54:18.413412 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b2a45c-ff1a-47a6-9ff0-96db4a970701 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:18.423768 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b2a45c-ff1a-47a6-9ff0-96db4a970701 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:18.483392 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f059e58-aa72-4986-820a-b1f549112017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:18.484117 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f059e58-aa72-4986-820a-b1f549112017 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:18.484391 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f059e58-aa72-4986-820a-b1f549112017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:18.484712 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f059e58-aa72-4986-820a-b1f549112017 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:18.485596 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f059e58-aa72-4986-820a-b1f549112017 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:18.487507 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f059e58-aa72-4986-820a-b1f549112017 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:18.515223 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1301/2616] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:54:18 2022] POST /identity/v3/users => generated 291 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:54:18.516624 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f69fd1-adcd-4112-92ce-56973e84ec35 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:18.527173 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f69fd1-adcd-4112-92ce-56973e84ec35 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:18.532112 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1831883c-1d87-4aa7-a557-80fb79c3c3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:18.532808 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831883c-1d87-4aa7-a557-80fb79c3c3f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:18.533280 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831883c-1d87-4aa7-a557-80fb79c3c3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:18.533410 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831883c-1d87-4aa7-a557-80fb79c3c3f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:18.534135 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1831883c-1d87-4aa7-a557-80fb79c3c3f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:18.535744 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1831883c-1d87-4aa7-a557-80fb79c3c3f9 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:18.550360 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1316/2617] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:54:18.551823 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f059e58-aa72-4986-820a-b1f549112017 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:18.564414 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f059e58-aa72-4986-820a-b1f549112017 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:18.570155 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c08657e-179c-4090-a03b-37b3562de59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:18.571091 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08657e-179c-4090-a03b-37b3562de59a admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:18.571259 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08657e-179c-4090-a03b-37b3562de59a admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:18.571664 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08657e-179c-4090-a03b-37b3562de59a admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:18.572920 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c08657e-179c-4090-a03b-37b3562de59a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:18.574642 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c08657e-179c-4090-a03b-37b3562de59a admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:18.577141 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:54:18.596314 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1302/2618] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:54:18.602708 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1831883c-1d87-4aa7-a557-80fb79c3c3f9 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:18.630695 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1831883c-1d87-4aa7-a557-80fb79c3c3f9 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:18.636013 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4425dae-edd4-4117-8854-39d9d0d674d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:18.636920 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4425dae-edd4-4117-8854-39d9d0d674d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:18.637237 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4425dae-edd4-4117-8854-39d9d0d674d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:18.637520 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4425dae-edd4-4117-8854-39d9d0d674d8 admin admin] PATH_INFO: `/v3/projects/faf66cbbfe654621814279c00adf1ac0/users/1641da87e05b442bbbba33c05857c951/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:18.658513 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4425dae-edd4-4117-8854-39d9d0d674d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=faf66cbbfe654621814279c00adf1ac0, user_id=1641da87e05b442bbbba33c05857c951, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:18.661339 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4425dae-edd4-4117-8854-39d9d0d674d8 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:18.689329 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1317/2619] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:54:18 2022] PUT /identity/v3/projects/faf66cbbfe654621814279c00adf1ac0/users/1641da87e05b442bbbba33c05857c951/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:54:18.700371 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f6f4ad-b0d9-4621-a172-07cfea074bf3 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:18.700671 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f6f4ad-b0d9-4621-a172-07cfea074bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:18.700950 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f6f4ad-b0d9-4621-a172-07cfea074bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:18.736858 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-57f6f4ad-b0d9-4621-a172-07cfea074bf3 None None] MFA Rules not processed for user `1641da87e05b442bbbba33c05857c951`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:54:18.738124 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-57f6f4ad-b0d9-4621-a172-07cfea074bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:18.802931 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1303/2620] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:54:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:54:18.807462 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4425dae-edd4-4117-8854-39d9d0d674d8 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:18.878098 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4425dae-edd4-4117-8854-39d9d0d674d8 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:18.882790 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdb818a-340f-43c0-8c11-c1ae2813a225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:18.883387 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb818a-340f-43c0-8c11-c1ae2813a225 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:18.883676 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb818a-340f-43c0-8c11-c1ae2813a225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:18.883959 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb818a-340f-43c0-8c11-c1ae2813a225 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:18.884646 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdb818a-340f-43c0-8c11-c1ae2813a225 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:18.886626 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdb818a-340f-43c0-8c11-c1ae2813a225 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:18.912846 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1318/2621] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:54:18 2022] POST /identity/v3/projects => generated 381 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:54:18.916911 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f6f4ad-b0d9-4621-a172-07cfea074bf3 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:18.938197 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f6f4ad-b0d9-4621-a172-07cfea074bf3 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:18.993497 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa852f-a4f9-40ab-b49c-2bd9a539cf5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:18.994163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa852f-a4f9-40ab-b49c-2bd9a539cf5f admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:18.994448 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa852f-a4f9-40ab-b49c-2bd9a539cf5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:18.994725 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa852f-a4f9-40ab-b49c-2bd9a539cf5f admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:18.995417 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfa852f-a4f9-40ab-b49c-2bd9a539cf5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:18.997600 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfa852f-a4f9-40ab-b49c-2bd9a539cf5f admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:19.016347 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1304/2622] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:54:18 2022] POST /identity/v3/users => generated 297 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:54:19.020343 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdb818a-340f-43c0-8c11-c1ae2813a225 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:19.031263 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdb818a-340f-43c0-8c11-c1ae2813a225 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:19.036146 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29922957-b614-4f71-b08f-8d453ac4f11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:19.037432 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29922957-b614-4f71-b08f-8d453ac4f11f admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:19.037725 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29922957-b614-4f71-b08f-8d453ac4f11f admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:19.038007 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29922957-b614-4f71-b08f-8d453ac4f11f admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:19.038745 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29922957-b614-4f71-b08f-8d453ac4f11f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:19.040424 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29922957-b614-4f71-b08f-8d453ac4f11f admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:19.050465 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1319/2623] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:54:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:54:19.054407 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa852f-a4f9-40ab-b49c-2bd9a539cf5f admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:19.064299 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa852f-a4f9-40ab-b49c-2bd9a539cf5f admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:19.069440 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd15414-5218-4bd3-8373-07b48b6ae0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:19.070317 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd15414-5218-4bd3-8373-07b48b6ae0d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:19.070603 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd15414-5218-4bd3-8373-07b48b6ae0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:19.070884 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd15414-5218-4bd3-8373-07b48b6ae0d3 admin admin] PATH_INFO: `/v3/projects/33918b17a6164bb3a8038132d56ec616/users/47859931a0984d408b6d4aef4985138d/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:19.086576 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd15414-5218-4bd3-8373-07b48b6ae0d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33918b17a6164bb3a8038132d56ec616, user_id=47859931a0984d408b6d4aef4985138d, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:19.089236 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd15414-5218-4bd3-8373-07b48b6ae0d3 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:19.101521 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1305/2624] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:54:19 2022] PUT /identity/v3/projects/33918b17a6164bb3a8038132d56ec616/users/47859931a0984d408b6d4aef4985138d/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:54:19.105415 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29922957-b614-4f71-b08f-8d453ac4f11f admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:19.115291 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29922957-b614-4f71-b08f-8d453ac4f11f admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:19.170563 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c174cab-f69a-4540-a05f-ff17f4d89ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:19.171156 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c174cab-f69a-4540-a05f-ff17f4d89ec7 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:19.171451 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c174cab-f69a-4540-a05f-ff17f4d89ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:19.171726 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c174cab-f69a-4540-a05f-ff17f4d89ec7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:19.172448 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c174cab-f69a-4540-a05f-ff17f4d89ec7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:19.174096 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c174cab-f69a-4540-a05f-ff17f4d89ec7 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:19.176128 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:54:19.188185 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1320/2625] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:54:19 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:54:19.189584 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd15414-5218-4bd3-8373-07b48b6ae0d3 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:19.200393 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd15414-5218-4bd3-8373-07b48b6ae0d3 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:19.204944 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a0cd0-d904-475c-ae1d-7e898b88b18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:19.205845 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a0cd0-d904-475c-ae1d-7e898b88b18c admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:19.206128 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a0cd0-d904-475c-ae1d-7e898b88b18c admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:19.206404 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a0cd0-d904-475c-ae1d-7e898b88b18c admin admin] PATH_INFO: `/v3/projects/33918b17a6164bb3a8038132d56ec616/users/47859931a0984d408b6d4aef4985138d/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:19.208322 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64a0cd0-d904-475c-ae1d-7e898b88b18c admin admin] RBAC: Authorizing `identity:create_grant(project_id=33918b17a6164bb3a8038132d56ec616, user_id=47859931a0984d408b6d4aef4985138d, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:19.211562 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64a0cd0-d904-475c-ae1d-7e898b88b18c admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:19.221373 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1306/2626] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:54:19 2022] PUT /identity/v3/projects/33918b17a6164bb3a8038132d56ec616/users/47859931a0984d408b6d4aef4985138d/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:54:19.231525 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2300f69-1517-4b07-8f8d-ca3fd45e675b None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:19.231843 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2300f69-1517-4b07-8f8d-ca3fd45e675b None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:19.232131 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2300f69-1517-4b07-8f8d-ca3fd45e675b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:19.260515 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-b2300f69-1517-4b07-8f8d-ca3fd45e675b None None] MFA Rules not processed for user `47859931a0984d408b6d4aef4985138d`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:54:19.261794 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b2300f69-1517-4b07-8f8d-ca3fd45e675b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:19.328761 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1321/2627] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:54:19 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:54:19.340472 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a0cd0-d904-475c-ae1d-7e898b88b18c admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:19.342192 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-f64a0cd0-d904-475c-ae1d-7e898b88b18c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:19.364155 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:54:19.399944 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a0cd0-d904-475c-ae1d-7e898b88b18c admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:19.403576 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9bebe8-6b9f-466c-88fd-bb58b1a0eee6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:19.404214 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9bebe8-6b9f-466c-88fd-bb58b1a0eee6 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:19.404504 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9bebe8-6b9f-466c-88fd-bb58b1a0eee6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:19.404786 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9bebe8-6b9f-466c-88fd-bb58b1a0eee6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:19.413747 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9bebe8-6b9f-466c-88fd-bb58b1a0eee6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:19.415179 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9bebe8-6b9f-466c-88fd-bb58b1a0eee6 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:19.424769 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1307/2628] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:54:19 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:54:19.526743 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2300f69-1517-4b07-8f8d-ca3fd45e675b None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:19.550495 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2300f69-1517-4b07-8f8d-ca3fd45e675b None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:19.554636 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a79c72-9ff2-406a-a1a1-ba3307f33097 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:19.555303 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a79c72-9ff2-406a-a1a1-ba3307f33097 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:19.555589 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a79c72-9ff2-406a-a1a1-ba3307f33097 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:19.555879 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a79c72-9ff2-406a-a1a1-ba3307f33097 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:19.611478 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a79c72-9ff2-406a-a1a1-ba3307f33097 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:19.612931 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a79c72-9ff2-406a-a1a1-ba3307f33097 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:19.622780 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1322/2629] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:54:19 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:54:21.156372 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9bebe8-6b9f-466c-88fd-bb58b1a0eee6 service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:21.158135 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ee9bebe8-6b9f-466c-88fd-bb58b1a0eee6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:21.244877 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9bebe8-6b9f-466c-88fd-bb58b1a0eee6 service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:21.250646 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240a666-0f1a-490c-8f9f-faf9753f4586 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:21.251289 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240a666-0f1a-490c-8f9f-faf9753f4586 service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:21.251577 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240a666-0f1a-490c-8f9f-faf9753f4586 service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:21.251858 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240a666-0f1a-490c-8f9f-faf9753f4586 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:21.253666 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-c240a666-0f1a-490c-8f9f-faf9753f4586 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:21.331744 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c240a666-0f1a-490c-8f9f-faf9753f4586 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:21.333370 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c240a666-0f1a-490c-8f9f-faf9753f4586 service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:21.351596 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1308/2630] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:54:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:54:26.387186 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a79c72-9ff2-406a-a1a1-ba3307f33097 service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:26.451909 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a79c72-9ff2-406a-a1a1-ba3307f33097 service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:26.456501 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a2bd6c-753b-4ccc-8694-fbb9914ef656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:26.457394 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a2bd6c-753b-4ccc-8694-fbb9914ef656 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:26.457686 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a2bd6c-753b-4ccc-8694-fbb9914ef656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:26.457973 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a2bd6c-753b-4ccc-8694-fbb9914ef656 admin admin] PATH_INFO: `/v3/users/1641da87e05b442bbbba33c05857c951` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:26.459534 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a2bd6c-753b-4ccc-8694-fbb9914ef656 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1641da87e05b442bbbba33c05857c951)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:26.461435 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a2bd6c-753b-4ccc-8694-fbb9914ef656 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:26.477395 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:54:26.519094 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-48a2bd6c-753b-4ccc-8694-fbb9914ef656 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1641da87e05b442bbbba33c05857c951'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:26.519872 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-48a2bd6c-753b-4ccc-8694-fbb9914ef656 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1641da87e05b442bbbba33c05857c951'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:26.520481 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-48a2bd6c-753b-4ccc-8694-fbb9914ef656 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1641da87e05b442bbbba33c05857c951'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:26.528422 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-48a2bd6c-753b-4ccc-8694-fbb9914ef656 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1641da87e05b442bbbba33c05857c951'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:26.541779 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-48a2bd6c-753b-4ccc-8694-fbb9914ef656 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1641da87e05b442bbbba33c05857c951'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:26.554386 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1323/2631] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:54:26 2022] DELETE /identity/v3/users/1641da87e05b442bbbba33c05857c951 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:54:26.558836 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08246f-755d-48bc-ab9a-25250cb51d03 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:26.559329 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08246f-755d-48bc-ab9a-25250cb51d03 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:26.559460 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08246f-755d-48bc-ab9a-25250cb51d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:26.585277 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-4f08246f-755d-48bc-ab9a-25250cb51d03 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:54:26.586646 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-4f08246f-755d-48bc-ab9a-25250cb51d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:26.590772 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:54:26.645339 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1309/2632] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:54:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:54:26.655944 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a2bd6c-753b-4ccc-8694-fbb9914ef656 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:26.657356 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-48a2bd6c-753b-4ccc-8694-fbb9914ef656 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:26.741288 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a2bd6c-753b-4ccc-8694-fbb9914ef656 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:26.745878 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d4b8d8-621a-4193-9e08-792ae2f51f1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:26.747043 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d4b8d8-621a-4193-9e08-792ae2f51f1c service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:26.747741 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d4b8d8-621a-4193-9e08-792ae2f51f1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:26.748240 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d4b8d8-621a-4193-9e08-792ae2f51f1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:26.759097 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d4b8d8-621a-4193-9e08-792ae2f51f1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:26.760692 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d4b8d8-621a-4193-9e08-792ae2f51f1c service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:26.770469 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1324/2633] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:54:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:54:27.043460 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f08246f-755d-48bc-ab9a-25250cb51d03 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:27.044970 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-4f08246f-755d-48bc-ab9a-25250cb51d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:27.059776 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f08246f-755d-48bc-ab9a-25250cb51d03 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:27.064871 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55bea44-21cb-42e3-9c79-af6316d6c967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:27.065674 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55bea44-21cb-42e3-9c79-af6316d6c967 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:27.065962 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55bea44-21cb-42e3-9c79-af6316d6c967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:27.066369 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55bea44-21cb-42e3-9c79-af6316d6c967 admin admin] PATH_INFO: `/v3/projects/faf66cbbfe654621814279c00adf1ac0` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:27.067692 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55bea44-21cb-42e3-9c79-af6316d6c967 admin admin] RBAC: Authorizing `identity:delete_project(project_id=faf66cbbfe654621814279c00adf1ac0)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:27.071080 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55bea44-21cb-42e3-9c79-af6316d6c967 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:27.116937 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-e55bea44-21cb-42e3-9c79-af6316d6c967 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'faf66cbbfe654621814279c00adf1ac0', 'request_id': 'req-e55bea44-21cb-42e3-9c79-af6316d6c967'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:27.122722 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-e55bea44-21cb-42e3-9c79-af6316d6c967 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'faf66cbbfe654621814279c00adf1ac0', 'request_id': 'req-e55bea44-21cb-42e3-9c79-af6316d6c967'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:27.128833 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-e55bea44-21cb-42e3-9c79-af6316d6c967 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'faf66cbbfe654621814279c00adf1ac0', 'request_id': 'req-e55bea44-21cb-42e3-9c79-af6316d6c967'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:27.141374 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-e55bea44-21cb-42e3-9c79-af6316d6c967 admin admin] The token cache is being invalidate because project faf66cbbfe654621814279c00adf1ac0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:54:27.141663 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-e55bea44-21cb-42e3-9c79-af6316d6c967 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:27.142276 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-e55bea44-21cb-42e3-9c79-af6316d6c967 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:27.143861 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1310/2634] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:54:27 2022] DELETE /identity/v3/projects/faf66cbbfe654621814279c00adf1ac0 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:54:27.148433 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d4b8d8-621a-4193-9e08-792ae2f51f1c service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:27.158036 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d4b8d8-621a-4193-9e08-792ae2f51f1c service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:27.215670 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e052c17-8d49-4c45-ac67-98dd12292662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:27.216539 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e052c17-8d49-4c45-ac67-98dd12292662 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:27.216826 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e052c17-8d49-4c45-ac67-98dd12292662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:27.217103 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e052c17-8d49-4c45-ac67-98dd12292662 admin admin] PATH_INFO: `/v3/users/47859931a0984d408b6d4aef4985138d` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:27.218328 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e052c17-8d49-4c45-ac67-98dd12292662 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47859931a0984d408b6d4aef4985138d)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:27.220246 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e052c17-8d49-4c45-ac67-98dd12292662 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:27.240540 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:54:27.265551 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-4e052c17-8d49-4c45-ac67-98dd12292662 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47859931a0984d408b6d4aef4985138d'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:27.266193 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-4e052c17-8d49-4c45-ac67-98dd12292662 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47859931a0984d408b6d4aef4985138d'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:27.266804 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-4e052c17-8d49-4c45-ac67-98dd12292662 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47859931a0984d408b6d4aef4985138d'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:27.274200 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-4e052c17-8d49-4c45-ac67-98dd12292662 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47859931a0984d408b6d4aef4985138d'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:27.285745 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-4e052c17-8d49-4c45-ac67-98dd12292662 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47859931a0984d408b6d4aef4985138d'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:27.297640 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1325/2635] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:54:27 2022] DELETE /identity/v3/users/47859931a0984d408b6d4aef4985138d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:54:27.569182 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55bea44-21cb-42e3-9c79-af6316d6c967 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:27.570684 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-e55bea44-21cb-42e3-9c79-af6316d6c967 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:27.604158 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55bea44-21cb-42e3-9c79-af6316d6c967 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:27.659238 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9476b2-30c3-4df5-90ee-518e520b281a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:27.660033 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9476b2-30c3-4df5-90ee-518e520b281a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:27.660320 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9476b2-30c3-4df5-90ee-518e520b281a admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:27.660627 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9476b2-30c3-4df5-90ee-518e520b281a admin admin] PATH_INFO: `/v3/projects/33918b17a6164bb3a8038132d56ec616` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:27.661865 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9476b2-30c3-4df5-90ee-518e520b281a admin admin] RBAC: Authorizing `identity:delete_project(project_id=33918b17a6164bb3a8038132d56ec616)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:27.663758 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9476b2-30c3-4df5-90ee-518e520b281a admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:27.680366 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:54:27.711049 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8c9476b2-30c3-4df5-90ee-518e520b281a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33918b17a6164bb3a8038132d56ec616', 'request_id': 'req-8c9476b2-30c3-4df5-90ee-518e520b281a'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:27.716558 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8c9476b2-30c3-4df5-90ee-518e520b281a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33918b17a6164bb3a8038132d56ec616', 'request_id': 'req-8c9476b2-30c3-4df5-90ee-518e520b281a'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:27.722589 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8c9476b2-30c3-4df5-90ee-518e520b281a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33918b17a6164bb3a8038132d56ec616', 'request_id': 'req-8c9476b2-30c3-4df5-90ee-518e520b281a'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:27.736157 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8c9476b2-30c3-4df5-90ee-518e520b281a admin admin] The token cache is being invalidate because project 33918b17a6164bb3a8038132d56ec616 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:54:27.736451 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8c9476b2-30c3-4df5-90ee-518e520b281a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:27.737080 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8c9476b2-30c3-4df5-90ee-518e520b281a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:27.738634 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1311/2636] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:54:27 2022] DELETE /identity/v3/projects/33918b17a6164bb3a8038132d56ec616 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:54:27.744421 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f4d3a-ba2d-43f5-991c-a106d6f66ed6 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:27.744903 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f4d3a-ba2d-43f5-991c-a106d6f66ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:27.744999 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f4d3a-ba2d-43f5-991c-a106d6f66ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:27.772151 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-710f4d3a-ba2d-43f5-991c-a106d6f66ed6 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:54:27.773388 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-710f4d3a-ba2d-43f5-991c-a106d6f66ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:27.794953 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9476b2-30c3-4df5-90ee-518e520b281a admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:27.796651 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-8c9476b2-30c3-4df5-90ee-518e520b281a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:27.834365 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1326/2637] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:54:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:54:27.842106 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47160fad-4c26-45ce-a1e2-5510a534e753 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:27.842520 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47160fad-4c26-45ce-a1e2-5510a534e753 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:27.842813 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47160fad-4c26-45ce-a1e2-5510a534e753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:27.861472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-47160fad-4c26-45ce-a1e2-5510a534e753 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:54:27.862714 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-47160fad-4c26-45ce-a1e2-5510a534e753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:27.863901 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9476b2-30c3-4df5-90ee-518e520b281a admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:27.869275 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0accf707-8e5e-4449-adcf-f79c446a2b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:27.870153 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0accf707-8e5e-4449-adcf-f79c446a2b35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:27.870494 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0accf707-8e5e-4449-adcf-f79c446a2b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:27.870782 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0accf707-8e5e-4449-adcf-f79c446a2b35 admin admin] PATH_INFO: `/v3/users/367eca85f09544aaa20741d1bfa7ff14` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:27.871430 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1327/2638] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:54:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:54:27.872251 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0accf707-8e5e-4449-adcf-f79c446a2b35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=367eca85f09544aaa20741d1bfa7ff14)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:27.874225 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0accf707-8e5e-4449-adcf-f79c446a2b35 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:27.875694 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47160fad-4c26-45ce-a1e2-5510a534e753 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:27.884634 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47160fad-4c26-45ce-a1e2-5510a534e753 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:27.889370 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac99c9a-14b1-4682-9a23-e4202d71bfc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:27.890027 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac99c9a-14b1-4682-9a23-e4202d71bfc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:27.890318 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac99c9a-14b1-4682-9a23-e4202d71bfc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:27.890604 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac99c9a-14b1-4682-9a23-e4202d71bfc4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:27.891353 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac99c9a-14b1-4682-9a23-e4202d71bfc4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:54:27.891667 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac99c9a-14b1-4682-9a23-e4202d71bfc4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:27.893351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac99c9a-14b1-4682-9a23-e4202d71bfc4 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:27.906513 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1328/2639] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:54:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:54:27.911196 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac99c9a-14b1-4682-9a23-e4202d71bfc4 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:27.921739 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac99c9a-14b1-4682-9a23-e4202d71bfc4 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:27.929497 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0accf707-8e5e-4449-adcf-f79c446a2b35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '367eca85f09544aaa20741d1bfa7ff14'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:27.930203 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0accf707-8e5e-4449-adcf-f79c446a2b35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '367eca85f09544aaa20741d1bfa7ff14'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:27.931122 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8f05fe-429b-4594-8155-cdb3ea0d0ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:27.931884 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f05fe-429b-4594-8155-cdb3ea0d0ba0 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:27.932068 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f05fe-429b-4594-8155-cdb3ea0d0ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:27.932349 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f05fe-429b-4594-8155-cdb3ea0d0ba0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:27.933048 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8f05fe-429b-4594-8155-cdb3ea0d0ba0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:27.935111 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8f05fe-429b-4594-8155-cdb3ea0d0ba0 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:27.938359 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0accf707-8e5e-4449-adcf-f79c446a2b35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '367eca85f09544aaa20741d1bfa7ff14'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:27.950981 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0accf707-8e5e-4449-adcf-f79c446a2b35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '367eca85f09544aaa20741d1bfa7ff14'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:27.951697 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-0accf707-8e5e-4449-adcf-f79c446a2b35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '367eca85f09544aaa20741d1bfa7ff14'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:27.962476 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1329/2640] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:54:27 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:54:27.965593 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1312/2641] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:54:27 2022] DELETE /identity/v3/users/367eca85f09544aaa20741d1bfa7ff14 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:54:27.966868 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8f05fe-429b-4594-8155-cdb3ea0d0ba0 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:27.968692 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-4f8f05fe-429b-4594-8155-cdb3ea0d0ba0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:27.969861 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb2a3b9-3496-412c-b34b-2af573e255ff None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:27.970151 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb2a3b9-3496-412c-b34b-2af573e255ff None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:27.970433 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb2a3b9-3496-412c-b34b-2af573e255ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:27.989197 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-8fb2a3b9-3496-412c-b34b-2af573e255ff None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:54:27.990966 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-8fb2a3b9-3496-412c-b34b-2af573e255ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:28.009542 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:54:28.021505 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:54:28.040280 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8f05fe-429b-4594-8155-cdb3ea0d0ba0 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:28.053780 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1313/2642] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:54:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:54:28.063824 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb2a3b9-3496-412c-b34b-2af573e255ff None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:28.065624 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-8fb2a3b9-3496-412c-b34b-2af573e255ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:28.099328 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6caa89c-f5a3-4cb2-bec0-980e4b068957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:28.100511 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6caa89c-f5a3-4cb2-bec0-980e4b068957 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:28.100803 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6caa89c-f5a3-4cb2-bec0-980e4b068957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:28.101111 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6caa89c-f5a3-4cb2-bec0-980e4b068957 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:28.101828 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6caa89c-f5a3-4cb2-bec0-980e4b068957 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:28.103787 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6caa89c-f5a3-4cb2-bec0-980e4b068957 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:28.127623 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1330/2643] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:54:27 2022] POST /identity/v3/users => generated 293 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:54:28.127814 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb2a3b9-3496-412c-b34b-2af573e255ff None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:28.128615 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6caa89c-f5a3-4cb2-bec0-980e4b068957 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:28.128734 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a109e61-de57-4591-998f-b5e1f75afa26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:28.129350 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a109e61-de57-4591-998f-b5e1f75afa26 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:28.129641 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a109e61-de57-4591-998f-b5e1f75afa26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:28.129922 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a109e61-de57-4591-998f-b5e1f75afa26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:28.138143 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6caa89c-f5a3-4cb2-bec0-980e4b068957 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:28.139253 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a109e61-de57-4591-998f-b5e1f75afa26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:28.140712 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a109e61-de57-4591-998f-b5e1f75afa26 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:28.142940 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb84470-a704-4d28-9e66-d2cf20e9519c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:28.143589 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb84470-a704-4d28-9e66-d2cf20e9519c admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:28.143887 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb84470-a704-4d28-9e66-d2cf20e9519c admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:28.144168 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb84470-a704-4d28-9e66-d2cf20e9519c admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:28.144905 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb84470-a704-4d28-9e66-d2cf20e9519c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:28.147092 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb84470-a704-4d28-9e66-d2cf20e9519c admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:28.150887 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1314/2644] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:54:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 16:54:28.157175 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1331/2645] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:54:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:54:28.161026 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a109e61-de57-4591-998f-b5e1f75afa26 service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:28.171364 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a109e61-de57-4591-998f-b5e1f75afa26 service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:28.175904 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76db4969-f8b5-47a2-b124-b237cbbd2219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:28.176572 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db4969-f8b5-47a2-b124-b237cbbd2219 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:28.176859 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db4969-f8b5-47a2-b124-b237cbbd2219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:28.177141 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db4969-f8b5-47a2-b124-b237cbbd2219 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:28.177931 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76db4969-f8b5-47a2-b124-b237cbbd2219 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:28.179562 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76db4969-f8b5-47a2-b124-b237cbbd2219 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:28.189948 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1315/2646] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:54:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:54:28.193880 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb84470-a704-4d28-9e66-d2cf20e9519c admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:28.204080 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb84470-a704-4d28-9e66-d2cf20e9519c admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:28.209391 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d147bd39-72ea-4abf-81d6-894b4a0d744b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:28.210401 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d147bd39-72ea-4abf-81d6-894b4a0d744b admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:28.210790 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d147bd39-72ea-4abf-81d6-894b4a0d744b admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:28.211169 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d147bd39-72ea-4abf-81d6-894b4a0d744b admin admin] PATH_INFO: `/v3/projects/85328f009e62419b9badda7c05cc9667/users/e225ee1be68c48c7b977554443292e81/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:28.229653 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d147bd39-72ea-4abf-81d6-894b4a0d744b admin admin] RBAC: Authorizing `identity:create_grant(project_id=85328f009e62419b9badda7c05cc9667, user_id=e225ee1be68c48c7b977554443292e81, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:28.232457 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d147bd39-72ea-4abf-81d6-894b4a0d744b admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:28.248187 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1332/2647] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:54:28 2022] PUT /identity/v3/projects/85328f009e62419b9badda7c05cc9667/users/e225ee1be68c48c7b977554443292e81/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:54:28.258521 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbf0d6e-6345-42f1-bad1-65724a531071 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:28.258975 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbf0d6e-6345-42f1-bad1-65724a531071 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:28.259290 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbf0d6e-6345-42f1-bad1-65724a531071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:28.290534 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-ddbf0d6e-6345-42f1-bad1-65724a531071 None None] MFA Rules not processed for user `e225ee1be68c48c7b977554443292e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:54:28.291759 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ddbf0d6e-6345-42f1-bad1-65724a531071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:28.363185 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1316/2648] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:54:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:54:28.375674 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d147bd39-72ea-4abf-81d6-894b4a0d744b admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:28.377300 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d147bd39-72ea-4abf-81d6-894b4a0d744b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:28.436711 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d147bd39-72ea-4abf-81d6-894b4a0d744b admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:28.440583 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f736ed-3928-434d-81df-ebe2b5a56d2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:28.441558 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f736ed-3928-434d-81df-ebe2b5a56d2b service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:28.441935 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f736ed-3928-434d-81df-ebe2b5a56d2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:28.442259 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f736ed-3928-434d-81df-ebe2b5a56d2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:28.446111 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbf0d6e-6345-42f1-bad1-65724a531071 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:28.447760 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ddbf0d6e-6345-42f1-bad1-65724a531071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:28.452482 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f736ed-3928-434d-81df-ebe2b5a56d2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:28.454027 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f736ed-3928-434d-81df-ebe2b5a56d2b service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:28.464174 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1333/2649] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:54:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:54:28.487519 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f736ed-3928-434d-81df-ebe2b5a56d2b service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:28.504444 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=100 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:54:28.523440 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbf0d6e-6345-42f1-bad1-65724a531071 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:28.528010 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8990297-b42b-4939-a616-33984e8060bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:28.528999 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8990297-b42b-4939-a616-33984e8060bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:28.529325 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8990297-b42b-4939-a616-33984e8060bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:28.529612 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8990297-b42b-4939-a616-33984e8060bb admin admin] PATH_INFO: `/v3/projects/7511afdd52d341f9a1dad3b628654693` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:28.530810 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8990297-b42b-4939-a616-33984e8060bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=7511afdd52d341f9a1dad3b628654693)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:28.532727 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8990297-b42b-4939-a616-33984e8060bb admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:28.541628 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:54:28.553824 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f736ed-3928-434d-81df-ebe2b5a56d2b service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:28.558474 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c582563-6893-4a27-9c22-c92d7e7b4819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:28.559458 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c582563-6893-4a27-9c22-c92d7e7b4819 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:28.559665 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c582563-6893-4a27-9c22-c92d7e7b4819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:28.559943 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c582563-6893-4a27-9c22-c92d7e7b4819 admin admin] PATH_INFO: `/v3/users/e225ee1be68c48c7b977554443292e81` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:28.561296 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c582563-6893-4a27-9c22-c92d7e7b4819 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e225ee1be68c48c7b977554443292e81)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:28.563355 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c582563-6893-4a27-9c22-c92d7e7b4819 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:28.581015 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-c8990297-b42b-4939-a616-33984e8060bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7511afdd52d341f9a1dad3b628654693', 'request_id': 'req-c8990297-b42b-4939-a616-33984e8060bb'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:28.587185 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-c8990297-b42b-4939-a616-33984e8060bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7511afdd52d341f9a1dad3b628654693', 'request_id': 'req-c8990297-b42b-4939-a616-33984e8060bb'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:28.593548 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-c8990297-b42b-4939-a616-33984e8060bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7511afdd52d341f9a1dad3b628654693', 'request_id': 'req-c8990297-b42b-4939-a616-33984e8060bb'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:28.606275 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-c8990297-b42b-4939-a616-33984e8060bb admin admin] The token cache is being invalidate because project 7511afdd52d341f9a1dad3b628654693 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:54:28.606570 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-c8990297-b42b-4939-a616-33984e8060bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:28.607217 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-c8990297-b42b-4939-a616-33984e8060bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:28.608860 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1317/2650] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:54:28 2022] DELETE /identity/v3/projects/7511afdd52d341f9a1dad3b628654693 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:54:28.613337 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b10ac-6b43-4461-bd87-8619b8aef048 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:28.613626 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b10ac-6b43-4461-bd87-8619b8aef048 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:28.613909 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b10ac-6b43-4461-bd87-8619b8aef048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:28.616061 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-0c582563-6893-4a27-9c22-c92d7e7b4819 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e225ee1be68c48c7b977554443292e81'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:28.616716 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-0c582563-6893-4a27-9c22-c92d7e7b4819 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e225ee1be68c48c7b977554443292e81'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:28.617360 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-0c582563-6893-4a27-9c22-c92d7e7b4819 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e225ee1be68c48c7b977554443292e81'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:28.625130 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-0c582563-6893-4a27-9c22-c92d7e7b4819 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e225ee1be68c48c7b977554443292e81'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:28.633508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-5e2b10ac-6b43-4461-bd87-8619b8aef048 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:54:28.634830 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-5e2b10ac-6b43-4461-bd87-8619b8aef048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:28.637443 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-0c582563-6893-4a27-9c22-c92d7e7b4819 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e225ee1be68c48c7b977554443292e81'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:28.650860 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1334/2651] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:54:28 2022] DELETE /identity/v3/users/e225ee1be68c48c7b977554443292e81 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:54:28.655269 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44696fa-fb52-4e74-bf89-46effc1c13dd None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:28.655641 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44696fa-fb52-4e74-bf89-46effc1c13dd None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:28.655930 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44696fa-fb52-4e74-bf89-46effc1c13dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:28.672284 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-b44696fa-fb52-4e74-bf89-46effc1c13dd None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:54:28.673642 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b44696fa-fb52-4e74-bf89-46effc1c13dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:28.692511 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1318/2652] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:54:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:54:28.700339 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d810f5-fb5c-4464-aea4-822697110757 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:28.700634 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d810f5-fb5c-4464-aea4-822697110757 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:28.700921 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d810f5-fb5c-4464-aea4-822697110757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:28.718537 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-a5d810f5-fb5c-4464-aea4-822697110757 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:54:28.719726 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-a5d810f5-fb5c-4464-aea4-822697110757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:28.733179 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1335/2653] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:54:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:54:28.742755 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44696fa-fb52-4e74-bf89-46effc1c13dd None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:28.744074 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b44696fa-fb52-4e74-bf89-46effc1c13dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:28.779438 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1319/2654] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:54:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:54:28.783706 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d810f5-fb5c-4464-aea4-822697110757 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:28.794027 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d810f5-fb5c-4464-aea4-822697110757 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:28.798741 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a40eac-0bc4-44e5-95cd-2f50200e8a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:28.799385 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a40eac-0bc4-44e5-95cd-2f50200e8a8f admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:28.799672 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a40eac-0bc4-44e5-95cd-2f50200e8a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:28.799953 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a40eac-0bc4-44e5-95cd-2f50200e8a8f admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:28.800688 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a40eac-0bc4-44e5-95cd-2f50200e8a8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:54:28.801005 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a40eac-0bc4-44e5-95cd-2f50200e8a8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:28.802776 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a40eac-0bc4-44e5-95cd-2f50200e8a8f admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:28.804172 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44696fa-fb52-4e74-bf89-46effc1c13dd None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:28.807858 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d949b3b-89f5-4742-9828-adc4f2ce524b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:28.808509 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d949b3b-89f5-4742-9828-adc4f2ce524b service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:28.808801 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d949b3b-89f5-4742-9828-adc4f2ce524b service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:28.809084 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d949b3b-89f5-4742-9828-adc4f2ce524b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:28.814431 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1320/2655] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:54:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:54:28.819300 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a40eac-0bc4-44e5-95cd-2f50200e8a8f admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:28.819954 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d949b3b-89f5-4742-9828-adc4f2ce524b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:28.821432 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d949b3b-89f5-4742-9828-adc4f2ce524b service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:28.825316 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:54:28.829440 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a40eac-0bc4-44e5-95cd-2f50200e8a8f admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:28.833179 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1336/2656] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:54:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 16:54:28.839249 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53ef317-c127-4ef2-8afb-e0fa9e721e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:28.839912 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53ef317-c127-4ef2-8afb-e0fa9e721e23 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:28.840200 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53ef317-c127-4ef2-8afb-e0fa9e721e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:28.840479 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53ef317-c127-4ef2-8afb-e0fa9e721e23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:28.841217 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53ef317-c127-4ef2-8afb-e0fa9e721e23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:28.843111 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53ef317-c127-4ef2-8afb-e0fa9e721e23 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:28.874739 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1321/2657] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:54:28 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:54:28.875882 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d949b3b-89f5-4742-9828-adc4f2ce524b service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:28.886205 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d949b3b-89f5-4742-9828-adc4f2ce524b service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:28.904075 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:54:28.960984 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d8bb5f-bfbb-407b-aff1-1341b687cdd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:28.961619 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d8bb5f-bfbb-407b-aff1-1341b687cdd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:28.961903 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d8bb5f-bfbb-407b-aff1-1341b687cdd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:28.962180 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d8bb5f-bfbb-407b-aff1-1341b687cdd5 admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:28.962880 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d8bb5f-bfbb-407b-aff1-1341b687cdd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:28.965208 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d8bb5f-bfbb-407b-aff1-1341b687cdd5 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:28.988652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1337/2658] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:54:28 2022] POST /identity/v3/users => generated 287 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:54:28.993193 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53ef317-c127-4ef2-8afb-e0fa9e721e23 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:29.003897 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53ef317-c127-4ef2-8afb-e0fa9e721e23 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:29.009275 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e3607-bd04-43d6-8af7-1ed21499117e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:29.010219 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e3607-bd04-43d6-8af7-1ed21499117e admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:29.010361 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e3607-bd04-43d6-8af7-1ed21499117e admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:29.010684 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e3607-bd04-43d6-8af7-1ed21499117e admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:29.011474 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61e3607-bd04-43d6-8af7-1ed21499117e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:29.013136 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61e3607-bd04-43d6-8af7-1ed21499117e admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:29.025183 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1322/2659] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:54:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:54:29.027407 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d8bb5f-bfbb-407b-aff1-1341b687cdd5 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:29.037715 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d8bb5f-bfbb-407b-aff1-1341b687cdd5 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:29.042470 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd9e932-f1a1-476a-95ec-71abe4833b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:29.043201 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd9e932-f1a1-476a-95ec-71abe4833b40 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:29.043491 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd9e932-f1a1-476a-95ec-71abe4833b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:29.043840 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd9e932-f1a1-476a-95ec-71abe4833b40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:29.044633 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd9e932-f1a1-476a-95ec-71abe4833b40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:29.046361 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd9e932-f1a1-476a-95ec-71abe4833b40 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:29.057677 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1338/2660] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:54:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:54:29.061760 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e3607-bd04-43d6-8af7-1ed21499117e admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:29.072584 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e3607-bd04-43d6-8af7-1ed21499117e admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:29.077255 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e0d82a-cb59-4f68-bd58-68da518f0b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:29.078143 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0d82a-cb59-4f68-bd58-68da518f0b1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:29.078429 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0d82a-cb59-4f68-bd58-68da518f0b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:29.078705 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0d82a-cb59-4f68-bd58-68da518f0b1f admin admin] PATH_INFO: `/v3/projects/efe37371967543a3aa0834579ad5e714/users/8d0d2a7bb14c44a3b28a11274a3a351e/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:29.095147 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e0d82a-cb59-4f68-bd58-68da518f0b1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=efe37371967543a3aa0834579ad5e714, user_id=8d0d2a7bb14c44a3b28a11274a3a351e, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:29.098044 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e0d82a-cb59-4f68-bd58-68da518f0b1f admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:29.110507 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1323/2661] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:54:29 2022] PUT /identity/v3/projects/efe37371967543a3aa0834579ad5e714/users/8d0d2a7bb14c44a3b28a11274a3a351e/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:54:29.113038 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd9e932-f1a1-476a-95ec-71abe4833b40 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:29.115936 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a921e-1aba-472b-abfe-95107e722c13 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:29.116232 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-dfd9e932-f1a1-476a-95ec-71abe4833b40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:29.116375 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a921e-1aba-472b-abfe-95107e722c13 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:29.116576 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a921e-1aba-472b-abfe-95107e722c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:29.132855 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd9e932-f1a1-476a-95ec-71abe4833b40 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:29.137668 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-ae4a921e-1aba-472b-abfe-95107e722c13 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:54:29.139162 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ae4a921e-1aba-472b-abfe-95107e722c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:29.194708 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf77e58-2492-4fc5-ba32-c514d56afea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:29.195611 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf77e58-2492-4fc5-ba32-c514d56afea2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:29.195902 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf77e58-2492-4fc5-ba32-c514d56afea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:29.196295 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf77e58-2492-4fc5-ba32-c514d56afea2 admin admin] PATH_INFO: `/v3/projects/85328f009e62419b9badda7c05cc9667` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:29.197522 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf77e58-2492-4fc5-ba32-c514d56afea2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85328f009e62419b9badda7c05cc9667)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:29.199436 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf77e58-2492-4fc5-ba32-c514d56afea2 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:29.203170 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1324/2662] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:54:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:54:29.212802 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4a921e-1aba-472b-abfe-95107e722c13 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:29.214097 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ae4a921e-1aba-472b-abfe-95107e722c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:29.247929 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ccf77e58-2492-4fc5-ba32-c514d56afea2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85328f009e62419b9badda7c05cc9667', 'request_id': 'req-ccf77e58-2492-4fc5-ba32-c514d56afea2'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:29.253884 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ccf77e58-2492-4fc5-ba32-c514d56afea2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85328f009e62419b9badda7c05cc9667', 'request_id': 'req-ccf77e58-2492-4fc5-ba32-c514d56afea2'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:29.260031 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ccf77e58-2492-4fc5-ba32-c514d56afea2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85328f009e62419b9badda7c05cc9667', 'request_id': 'req-ccf77e58-2492-4fc5-ba32-c514d56afea2'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:29.272294 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ccf77e58-2492-4fc5-ba32-c514d56afea2 admin admin] The token cache is being invalidate because project 85328f009e62419b9badda7c05cc9667 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:54:29.272589 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ccf77e58-2492-4fc5-ba32-c514d56afea2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:29.273263 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ccf77e58-2492-4fc5-ba32-c514d56afea2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:29.273702 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4a921e-1aba-472b-abfe-95107e722c13 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:29.274833 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1339/2663] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:54:29 2022] DELETE /identity/v3/projects/85328f009e62419b9badda7c05cc9667 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:54:29.279609 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61420662-afd9-42d1-8c22-4a135e94f11d None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:29.279898 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61420662-afd9-42d1-8c22-4a135e94f11d None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:29.280179 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61420662-afd9-42d1-8c22-4a135e94f11d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:29.301692 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-61420662-afd9-42d1-8c22-4a135e94f11d None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:54:29.303571 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-61420662-afd9-42d1-8c22-4a135e94f11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:29.323400 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:54:29.326997 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449043d5-8304-4f6e-9d0e-ac4e41f21000 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:29.327661 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449043d5-8304-4f6e-9d0e-ac4e41f21000 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:29.327977 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449043d5-8304-4f6e-9d0e-ac4e41f21000 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:29.328260 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449043d5-8304-4f6e-9d0e-ac4e41f21000 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:29.330569 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-449043d5-8304-4f6e-9d0e-ac4e41f21000 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:29.351456 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:54:29.380310 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1340/2664] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:54:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:54:29.388661 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce5122-890d-4ef6-8b17-9eaae209fe46 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:29.389123 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce5122-890d-4ef6-8b17-9eaae209fe46 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:29.389586 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce5122-890d-4ef6-8b17-9eaae209fe46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:29.401447 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449043d5-8304-4f6e-9d0e-ac4e41f21000 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:29.403251 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449043d5-8304-4f6e-9d0e-ac4e41f21000 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:29.407440 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-0bce5122-890d-4ef6-8b17-9eaae209fe46 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:54:29.408627 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-0bce5122-890d-4ef6-8b17-9eaae209fe46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:29.413341 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1325/2665] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:54:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 16:54:29.417076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1341/2666] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:54:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:54:29.425061 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449043d5-8304-4f6e-9d0e-ac4e41f21000 service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:29.437128 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449043d5-8304-4f6e-9d0e-ac4e41f21000 service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:29.442648 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c311dba-20bd-4659-8bdf-518e28a35612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:29.443566 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c311dba-20bd-4659-8bdf-518e28a35612 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:29.443852 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c311dba-20bd-4659-8bdf-518e28a35612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:29.444163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c311dba-20bd-4659-8bdf-518e28a35612 admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:29.444890 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c311dba-20bd-4659-8bdf-518e28a35612 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:54:29.445264 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c311dba-20bd-4659-8bdf-518e28a35612 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:29.446906 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c311dba-20bd-4659-8bdf-518e28a35612 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:29.459363 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1326/2667] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:54:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:54:29.463691 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bce5122-890d-4ef6-8b17-9eaae209fe46 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:29.472965 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bce5122-890d-4ef6-8b17-9eaae209fe46 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:29.478389 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a315c0-70b4-4e03-a2a1-2a883dddb1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:29.479164 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a315c0-70b4-4e03-a2a1-2a883dddb1ba admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:29.479518 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a315c0-70b4-4e03-a2a1-2a883dddb1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:29.479890 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a315c0-70b4-4e03-a2a1-2a883dddb1ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:29.480693 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a315c0-70b4-4e03-a2a1-2a883dddb1ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:29.482743 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a315c0-70b4-4e03-a2a1-2a883dddb1ba admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:29.513955 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1342/2668] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:54:29 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:54:29.515531 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c311dba-20bd-4659-8bdf-518e28a35612 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:29.526451 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c311dba-20bd-4659-8bdf-518e28a35612 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:29.587507 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4942bd-da12-47f5-81fe-e6579350830d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:29.588197 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4942bd-da12-47f5-81fe-e6579350830d admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:29.588487 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4942bd-da12-47f5-81fe-e6579350830d admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:29.588769 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4942bd-da12-47f5-81fe-e6579350830d admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:29.590064 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4942bd-da12-47f5-81fe-e6579350830d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:29.591446 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4942bd-da12-47f5-81fe-e6579350830d admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:29.612141 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1327/2669] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:54:29 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:54:29.617097 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a315c0-70b4-4e03-a2a1-2a883dddb1ba admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:29.628539 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a315c0-70b4-4e03-a2a1-2a883dddb1ba admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:29.633422 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cfb620-1116-4e3b-9ca4-be8d27e91f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:29.634100 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cfb620-1116-4e3b-9ca4-be8d27e91f7a admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:29.634394 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cfb620-1116-4e3b-9ca4-be8d27e91f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:29.634668 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cfb620-1116-4e3b-9ca4-be8d27e91f7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:29.635404 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cfb620-1116-4e3b-9ca4-be8d27e91f7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:29.637453 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cfb620-1116-4e3b-9ca4-be8d27e91f7a admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:29.660207 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1343/2670] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:54:29 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:54:29.661387 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4942bd-da12-47f5-81fe-e6579350830d admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:29.672380 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4942bd-da12-47f5-81fe-e6579350830d admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:29.677254 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759d4426-4793-4edf-a3bb-6a7a75e4e286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:29.677958 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d4426-4793-4edf-a3bb-6a7a75e4e286 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:29.678251 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d4426-4793-4edf-a3bb-6a7a75e4e286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:29.678543 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d4426-4793-4edf-a3bb-6a7a75e4e286 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:29.679280 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759d4426-4793-4edf-a3bb-6a7a75e4e286 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:29.680887 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759d4426-4793-4edf-a3bb-6a7a75e4e286 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:29.691150 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1328/2671] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:54:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:54:29.695491 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cfb620-1116-4e3b-9ca4-be8d27e91f7a admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:29.705586 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cfb620-1116-4e3b-9ca4-be8d27e91f7a admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:29.710826 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd92b961-ff7e-43f2-9c16-2c4749347c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:29.711715 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92b961-ff7e-43f2-9c16-2c4749347c26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:29.712003 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92b961-ff7e-43f2-9c16-2c4749347c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:29.712288 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92b961-ff7e-43f2-9c16-2c4749347c26 admin admin] PATH_INFO: `/v3/projects/80b38c0c4f8542569d24fada9b39f6c8/users/dc88adf9e23c4e039bda69411cc61f91/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:29.732589 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd92b961-ff7e-43f2-9c16-2c4749347c26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80b38c0c4f8542569d24fada9b39f6c8, user_id=dc88adf9e23c4e039bda69411cc61f91, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:29.735333 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd92b961-ff7e-43f2-9c16-2c4749347c26 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:29.750175 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1344/2672] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:54:29 2022] PUT /identity/v3/projects/80b38c0c4f8542569d24fada9b39f6c8/users/dc88adf9e23c4e039bda69411cc61f91/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:54:29.760524 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5093131-1dd4-437b-911c-a330c768bd6e None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:29.760823 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5093131-1dd4-437b-911c-a330c768bd6e None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:29.761100 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5093131-1dd4-437b-911c-a330c768bd6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:29.790767 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-a5093131-1dd4-437b-911c-a330c768bd6e None None] MFA Rules not processed for user `dc88adf9e23c4e039bda69411cc61f91`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:54:29.792032 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-a5093131-1dd4-437b-911c-a330c768bd6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:29.862771 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1329/2673] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:54:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:54:29.874481 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd92b961-ff7e-43f2-9c16-2c4749347c26 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:29.876260 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-cd92b961-ff7e-43f2-9c16-2c4749347c26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:29.942303 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd92b961-ff7e-43f2-9c16-2c4749347c26 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:29.946500 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ec4d38-0c4f-4186-beae-47d9b8b70e69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:29.947161 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec4d38-0c4f-4186-beae-47d9b8b70e69 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:29.947950 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec4d38-0c4f-4186-beae-47d9b8b70e69 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:29.947950 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec4d38-0c4f-4186-beae-47d9b8b70e69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:29.976888 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ec4d38-0c4f-4186-beae-47d9b8b70e69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:29.978699 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ec4d38-0c4f-4186-beae-47d9b8b70e69 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:29.995197 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1345/2674] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:54:29 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:54:30.014782 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5093131-1dd4-437b-911c-a330c768bd6e None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:30.026068 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:54:30.099719 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5093131-1dd4-437b-911c-a330c768bd6e None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:30.100485 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91e187-b928-4590-96d3-7d7f3c1c7615 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:30.100843 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91e187-b928-4590-96d3-7d7f3c1c7615 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:30.102340 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91e187-b928-4590-96d3-7d7f3c1c7615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:30.105626 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b7fc9-d24b-4f65-ba0a-f00070085193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:30.106609 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b7fc9-d24b-4f65-ba0a-f00070085193 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:30.107003 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b7fc9-d24b-4f65-ba0a-f00070085193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:30.107386 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b7fc9-d24b-4f65-ba0a-f00070085193 admin admin] PATH_INFO: `/v3/users/dc88adf9e23c4e039bda69411cc61f91` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:30.108716 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95b7fc9-d24b-4f65-ba0a-f00070085193 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc88adf9e23c4e039bda69411cc61f91)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:30.111060 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95b7fc9-d24b-4f65-ba0a-f00070085193 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:30.136300 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-df91e187-b928-4590-96d3-7d7f3c1c7615 None None] MFA Rules not processed for user `8d0d2a7bb14c44a3b28a11274a3a351e`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:54:30.138098 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-df91e187-b928-4590-96d3-7d7f3c1c7615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:30.167489 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f95b7fc9-d24b-4f65-ba0a-f00070085193 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc88adf9e23c4e039bda69411cc61f91'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:30.168781 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f95b7fc9-d24b-4f65-ba0a-f00070085193 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc88adf9e23c4e039bda69411cc61f91'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:30.178240 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f95b7fc9-d24b-4f65-ba0a-f00070085193 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc88adf9e23c4e039bda69411cc61f91'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:30.196714 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f95b7fc9-d24b-4f65-ba0a-f00070085193 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc88adf9e23c4e039bda69411cc61f91'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:30.197495 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f95b7fc9-d24b-4f65-ba0a-f00070085193 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc88adf9e23c4e039bda69411cc61f91'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:30.206317 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:54:30.213061 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1330/2675] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:54:30 2022] DELETE /identity/v3/users/dc88adf9e23c4e039bda69411cc61f91 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:54:30.217658 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727fa00-846c-4b6e-8006-9729cfae07f1 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:30.218159 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727fa00-846c-4b6e-8006-9729cfae07f1 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:30.218458 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727fa00-846c-4b6e-8006-9729cfae07f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:30.220206 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1346/2676] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:54:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:54:30.231194 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91e187-b928-4590-96d3-7d7f3c1c7615 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:30.233091 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-df91e187-b928-4590-96d3-7d7f3c1c7615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:30.237374 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-8727fa00-846c-4b6e-8006-9729cfae07f1 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:54:30.238564 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-8727fa00-846c-4b6e-8006-9729cfae07f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:30.247162 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91e187-b928-4590-96d3-7d7f3c1c7615 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:30.305032 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ae46c-1351-4bc8-9dcd-8a7c47efabb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:30.305849 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ae46c-1351-4bc8-9dcd-8a7c47efabb6 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:30.306136 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ae46c-1351-4bc8-9dcd-8a7c47efabb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:30.306414 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ae46c-1351-4bc8-9dcd-8a7c47efabb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:30.319842 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1331/2677] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:54:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:54:30.331487 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8727fa00-846c-4b6e-8006-9729cfae07f1 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:30.332877 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-8727fa00-846c-4b6e-8006-9729cfae07f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:30.370373 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:54:30.378239 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385ae46c-1351-4bc8-9dcd-8a7c47efabb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:30.379719 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385ae46c-1351-4bc8-9dcd-8a7c47efabb6 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:30.390393 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1347/2678] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:54:30 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 16:54:30.409257 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8727fa00-846c-4b6e-8006-9729cfae07f1 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:30.412949 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e63d6f-84ed-41bd-8c49-012eeb32b3de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:30.413664 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e63d6f-84ed-41bd-8c49-012eeb32b3de service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:30.413957 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e63d6f-84ed-41bd-8c49-012eeb32b3de service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:30.414242 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e63d6f-84ed-41bd-8c49-012eeb32b3de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:30.423806 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e63d6f-84ed-41bd-8c49-012eeb32b3de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:30.425283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e63d6f-84ed-41bd-8c49-012eeb32b3de service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:30.435608 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1332/2679] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:54:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 16:54:30.736152 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ae46c-1351-4bc8-9dcd-8a7c47efabb6 service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:30.738222 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-385ae46c-1351-4bc8-9dcd-8a7c47efabb6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:30.769825 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ae46c-1351-4bc8-9dcd-8a7c47efabb6 service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:30.774448 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ee4e13-bf21-4c32-b777-bcd2f40dbc15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:30.775314 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee4e13-bf21-4c32-b777-bcd2f40dbc15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:30.775601 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee4e13-bf21-4c32-b777-bcd2f40dbc15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:30.775881 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee4e13-bf21-4c32-b777-bcd2f40dbc15 admin admin] PATH_INFO: `/v3/projects/80b38c0c4f8542569d24fada9b39f6c8` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:30.777074 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ee4e13-bf21-4c32-b777-bcd2f40dbc15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80b38c0c4f8542569d24fada9b39f6c8)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:30.779034 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ee4e13-bf21-4c32-b777-bcd2f40dbc15 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:30.829076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-55ee4e13-bf21-4c32-b777-bcd2f40dbc15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80b38c0c4f8542569d24fada9b39f6c8', 'request_id': 'req-55ee4e13-bf21-4c32-b777-bcd2f40dbc15'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:30.834696 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-55ee4e13-bf21-4c32-b777-bcd2f40dbc15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80b38c0c4f8542569d24fada9b39f6c8', 'request_id': 'req-55ee4e13-bf21-4c32-b777-bcd2f40dbc15'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:30.840654 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-55ee4e13-bf21-4c32-b777-bcd2f40dbc15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80b38c0c4f8542569d24fada9b39f6c8', 'request_id': 'req-55ee4e13-bf21-4c32-b777-bcd2f40dbc15'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:30.853701 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-55ee4e13-bf21-4c32-b777-bcd2f40dbc15 admin admin] The token cache is being invalidate because project 80b38c0c4f8542569d24fada9b39f6c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:54:30.853990 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-55ee4e13-bf21-4c32-b777-bcd2f40dbc15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:30.854606 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-55ee4e13-bf21-4c32-b777-bcd2f40dbc15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:30.856156 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1348/2680] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:54:30 2022] DELETE /identity/v3/projects/80b38c0c4f8542569d24fada9b39f6c8 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:54:30.861027 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b9459-fe6d-4c54-ab22-15d68d084f1f None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:30.861504 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b9459-fe6d-4c54-ab22-15d68d084f1f None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:30.861661 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b9459-fe6d-4c54-ab22-15d68d084f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:30.890389 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-a17b9459-fe6d-4c54-ab22-15d68d084f1f None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:54:30.891619 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-a17b9459-fe6d-4c54-ab22-15d68d084f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:30.949756 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1333/2681] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:54:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:54:30.957360 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7db0f9-0deb-4111-bac4-daef48b14585 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:30.957859 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7db0f9-0deb-4111-bac4-daef48b14585 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:30.957962 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7db0f9-0deb-4111-bac4-daef48b14585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:30.976670 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-1a7db0f9-0deb-4111-bac4-daef48b14585 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:54:30.977909 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-1a7db0f9-0deb-4111-bac4-daef48b14585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:30.985805 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1349/2682] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:54:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:54:30.989844 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17b9459-fe6d-4c54-ab22-15d68d084f1f None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:31.000172 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17b9459-fe6d-4c54-ab22-15d68d084f1f None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:31.004646 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d532ef26-e066-4e3b-86d2-5ab1afdbe973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:31.005357 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d532ef26-e066-4e3b-86d2-5ab1afdbe973 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:31.005645 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d532ef26-e066-4e3b-86d2-5ab1afdbe973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:31.005925 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d532ef26-e066-4e3b-86d2-5ab1afdbe973 admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:31.006651 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d532ef26-e066-4e3b-86d2-5ab1afdbe973 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:54:31.006989 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d532ef26-e066-4e3b-86d2-5ab1afdbe973 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:31.008602 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d532ef26-e066-4e3b-86d2-5ab1afdbe973 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:31.020004 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1334/2683] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:54:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:54:31.023981 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7db0f9-0deb-4111-bac4-daef48b14585 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:31.033053 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7db0f9-0deb-4111-bac4-daef48b14585 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:31.038308 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e98e9-ff5b-4f80-9573-97ab040f1d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:31.038958 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e98e9-ff5b-4f80-9573-97ab040f1d55 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:31.039263 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e98e9-ff5b-4f80-9573-97ab040f1d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:31.039526 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e98e9-ff5b-4f80-9573-97ab040f1d55 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:31.040221 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4e98e9-ff5b-4f80-9573-97ab040f1d55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:31.042216 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4e98e9-ff5b-4f80-9573-97ab040f1d55 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:31.067781 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1350/2684] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:54:31 2022] POST /identity/v3/projects => generated 363 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:54:31.071834 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d532ef26-e066-4e3b-86d2-5ab1afdbe973 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:31.081740 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d532ef26-e066-4e3b-86d2-5ab1afdbe973 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:31.137958 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05afa234-7ecd-42f7-a57c-a0cf6e5039d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:31.138626 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05afa234-7ecd-42f7-a57c-a0cf6e5039d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:31.138913 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05afa234-7ecd-42f7-a57c-a0cf6e5039d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:31.139191 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05afa234-7ecd-42f7-a57c-a0cf6e5039d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:31.139887 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05afa234-7ecd-42f7-a57c-a0cf6e5039d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:31.141845 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05afa234-7ecd-42f7-a57c-a0cf6e5039d1 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:31.161472 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1335/2685] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:54:31 2022] POST /identity/v3/users => generated 282 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:54:31.166086 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e98e9-ff5b-4f80-9573-97ab040f1d55 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:31.171571 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:54:31.177947 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e98e9-ff5b-4f80-9573-97ab040f1d55 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:31.183848 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef74c1c-bc2f-4cf0-b1a3-8163b68f3e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:31.184511 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef74c1c-bc2f-4cf0-b1a3-8163b68f3e42 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:31.184803 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef74c1c-bc2f-4cf0-b1a3-8163b68f3e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:31.185088 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef74c1c-bc2f-4cf0-b1a3-8163b68f3e42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:31.185866 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef74c1c-bc2f-4cf0-b1a3-8163b68f3e42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:31.187483 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef74c1c-bc2f-4cf0-b1a3-8163b68f3e42 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:31.197951 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1351/2686] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:54:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:54:31.201843 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05afa234-7ecd-42f7-a57c-a0cf6e5039d1 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:31.212104 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05afa234-7ecd-42f7-a57c-a0cf6e5039d1 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:31.216692 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7edde53-e84e-4574-9016-702effc7ce7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:31.217385 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7edde53-e84e-4574-9016-702effc7ce7b admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:31.217675 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7edde53-e84e-4574-9016-702effc7ce7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:31.217957 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7edde53-e84e-4574-9016-702effc7ce7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:31.218683 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7edde53-e84e-4574-9016-702effc7ce7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:31.220260 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7edde53-e84e-4574-9016-702effc7ce7b admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:31.230581 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1336/2687] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:54:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:54:31.234517 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef74c1c-bc2f-4cf0-b1a3-8163b68f3e42 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:31.244307 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef74c1c-bc2f-4cf0-b1a3-8163b68f3e42 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:31.249501 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a847d04-f24f-4381-b4c9-f90f7c21dbb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:31.250381 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a847d04-f24f-4381-b4c9-f90f7c21dbb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:31.250669 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a847d04-f24f-4381-b4c9-f90f7c21dbb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:31.250948 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a847d04-f24f-4381-b4c9-f90f7c21dbb5 admin admin] PATH_INFO: `/v3/projects/da1eaa88760a4f3bb9d2b90280b07ca2/users/d78b366335cd47a6a9ff44779496f7ea/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:31.266546 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a847d04-f24f-4381-b4c9-f90f7c21dbb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da1eaa88760a4f3bb9d2b90280b07ca2, user_id=d78b366335cd47a6a9ff44779496f7ea, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:31.269223 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a847d04-f24f-4381-b4c9-f90f7c21dbb5 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:31.283460 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1352/2688] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:54:31 2022] PUT /identity/v3/projects/da1eaa88760a4f3bb9d2b90280b07ca2/users/d78b366335cd47a6a9ff44779496f7ea/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:54:31.295990 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14681f00-0945-43a4-8cff-2fdcc83d86a0 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:31.296286 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14681f00-0945-43a4-8cff-2fdcc83d86a0 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:31.296567 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14681f00-0945-43a4-8cff-2fdcc83d86a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:31.322408 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-14681f00-0945-43a4-8cff-2fdcc83d86a0 None None] MFA Rules not processed for user `d78b366335cd47a6a9ff44779496f7ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:54:31.323626 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-14681f00-0945-43a4-8cff-2fdcc83d86a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:31.340171 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:54:31.388822 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1337/2689] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:54:31 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:54:31.400812 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a847d04-f24f-4381-b4c9-f90f7c21dbb5 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:31.402497 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-1a847d04-f24f-4381-b4c9-f90f7c21dbb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:31.461342 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a847d04-f24f-4381-b4c9-f90f7c21dbb5 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:31.464998 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3465fd04-2be5-4619-9d30-29909c83ccc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:31.465698 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3465fd04-2be5-4619-9d30-29909c83ccc4 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:31.465981 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3465fd04-2be5-4619-9d30-29909c83ccc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:31.466257 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3465fd04-2be5-4619-9d30-29909c83ccc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:31.474534 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3465fd04-2be5-4619-9d30-29909c83ccc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:31.476015 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3465fd04-2be5-4619-9d30-29909c83ccc4 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:31.485536 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1353/2690] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:54:31 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:54:31.760574 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136a72a-29d8-4d6f-acbf-acbc4fb0ca1c None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:31.760962 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136a72a-29d8-4d6f-acbf-acbc4fb0ca1c None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:31.761294 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136a72a-29d8-4d6f-acbf-acbc4fb0ca1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:31.781076 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-f136a72a-29d8-4d6f-acbf-acbc4fb0ca1c None None] MFA Rules not processed for user `8d0d2a7bb14c44a3b28a11274a3a351e`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:54:31.782551 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-f136a72a-29d8-4d6f-acbf-acbc4fb0ca1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:31.843157 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1338/2691] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:54:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:54:32.083227 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3465fd04-2be5-4619-9d30-29909c83ccc4 service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:32.084684 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-3465fd04-2be5-4619-9d30-29909c83ccc4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:32.146315 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3465fd04-2be5-4619-9d30-29909c83ccc4 service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:32.150361 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e827a-665f-4645-a90e-b51270688a87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:32.151056 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e827a-665f-4645-a90e-b51270688a87 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:32.151313 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e827a-665f-4645-a90e-b51270688a87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:32.151591 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e827a-665f-4645-a90e-b51270688a87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:32.153628 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-0e6e827a-665f-4645-a90e-b51270688a87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:32.164175 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6e827a-665f-4645-a90e-b51270688a87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:32.165763 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6e827a-665f-4645-a90e-b51270688a87 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:32.174241 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1354/2692] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:54:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:54:32.501004 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f136a72a-29d8-4d6f-acbf-acbc4fb0ca1c None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:32.502605 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-f136a72a-29d8-4d6f-acbf-acbc4fb0ca1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:32.589463 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f136a72a-29d8-4d6f-acbf-acbc4fb0ca1c None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:32.594716 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27901bfe-8260-4328-9147-1a9406b2a5d9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e158e5c87f4954aeb297f42dabdc16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:32.595450 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27901bfe-8260-4328-9147-1a9406b2a5d9 service placement] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:32.595808 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27901bfe-8260-4328-9147-1a9406b2a5d9 service placement] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:32.596093 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27901bfe-8260-4328-9147-1a9406b2a5d9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:32.597980 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-27901bfe-8260-4328-9147-1a9406b2a5d9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:32.610494 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27901bfe-8260-4328-9147-1a9406b2a5d9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:32.612079 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27901bfe-8260-4328-9147-1a9406b2a5d9 service placement] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:32.625554 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1339/2693] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:54:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:54:41.065831 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e827a-665f-4645-a90e-b51270688a87 service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:41.067805 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-0e6e827a-665f-4645-a90e-b51270688a87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:41.148934 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e827a-665f-4645-a90e-b51270688a87 service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:41.154492 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621e1b2f-bd0a-4035-a891-1d1beb800234 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:41.155183 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e1b2f-bd0a-4035-a891-1d1beb800234 service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:41.155470 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e1b2f-bd0a-4035-a891-1d1beb800234 service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:41.155755 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e1b2f-bd0a-4035-a891-1d1beb800234 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:41.157663 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-621e1b2f-bd0a-4035-a891-1d1beb800234 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:41.170490 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621e1b2f-bd0a-4035-a891-1d1beb800234 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:41.172126 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621e1b2f-bd0a-4035-a891-1d1beb800234 service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:41.176832 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:54:41.184150 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1355/2694] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:54:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:54:42.619340 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=76 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:54:49.382785 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27901bfe-8260-4328-9147-1a9406b2a5d9 service placement] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:49.392209 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27901bfe-8260-4328-9147-1a9406b2a5d9 service placement] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:49.395815 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572389f3-1243-492e-8a66-d66545c8a6a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:49.396471 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572389f3-1243-492e-8a66-d66545c8a6a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:49.397328 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572389f3-1243-492e-8a66-d66545c8a6a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:49.397704 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572389f3-1243-492e-8a66-d66545c8a6a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:49.399411 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-572389f3-1243-492e-8a66-d66545c8a6a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:49.465434 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572389f3-1243-492e-8a66-d66545c8a6a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:49.466882 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572389f3-1243-492e-8a66-d66545c8a6a0 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:49.476975 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1340/2695] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:54:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:54:51.184816 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:54:52.486480 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647ae4f-b649-4f88-9a2b-6c1694be7081 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:52.487047 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647ae4f-b649-4f88-9a2b-6c1694be7081 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:52.487173 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647ae4f-b649-4f88-9a2b-6c1694be7081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:52.507281 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-e647ae4f-b649-4f88-9a2b-6c1694be7081 None None] MFA Rules not processed for user `8d0d2a7bb14c44a3b28a11274a3a351e`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:54:52.508507 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e647ae4f-b649-4f88-9a2b-6c1694be7081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:52.515531 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1356/2696] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:54:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:54:53.362477 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572389f3-1243-492e-8a66-d66545c8a6a0 service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:53.364028 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-572389f3-1243-492e-8a66-d66545c8a6a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:53.382116 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572389f3-1243-492e-8a66-d66545c8a6a0 service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:53.388468 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd88162-911e-4272-afe3-a3c40a5c5813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:53.390533 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd88162-911e-4272-afe3-a3c40a5c5813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:53.390884 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd88162-911e-4272-afe3-a3c40a5c5813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:53.391175 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd88162-911e-4272-afe3-a3c40a5c5813 admin admin] PATH_INFO: `/v3/users/0695c868a2d844089fcda29ea3653d5c` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:53.392948 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd88162-911e-4272-afe3-a3c40a5c5813 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0695c868a2d844089fcda29ea3653d5c)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:53.395801 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd88162-911e-4272-afe3-a3c40a5c5813 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:53.489488 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-1fd88162-911e-4272-afe3-a3c40a5c5813 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0695c868a2d844089fcda29ea3653d5c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:53.490920 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-1fd88162-911e-4272-afe3-a3c40a5c5813 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0695c868a2d844089fcda29ea3653d5c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:53.499571 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-1fd88162-911e-4272-afe3-a3c40a5c5813 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0695c868a2d844089fcda29ea3653d5c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:53.512824 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-1fd88162-911e-4272-afe3-a3c40a5c5813 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0695c868a2d844089fcda29ea3653d5c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:53.513928 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-1fd88162-911e-4272-afe3-a3c40a5c5813 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0695c868a2d844089fcda29ea3653d5c'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:53.528769 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1341/2697] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:54:53 2022] DELETE /identity/v3/users/0695c868a2d844089fcda29ea3653d5c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:54:53.743348 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e647ae4f-b649-4f88-9a2b-6c1694be7081 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:53.757127 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e647ae4f-b649-4f88-9a2b-6c1694be7081 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:53.830112 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47dbcd6-2180-4093-9797-257706aceaa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:53.831351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47dbcd6-2180-4093-9797-257706aceaa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:53.831684 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47dbcd6-2180-4093-9797-257706aceaa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:53.831971 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47dbcd6-2180-4093-9797-257706aceaa7 admin admin] PATH_INFO: `/v3/projects/3e200a1b99664fc8b41757e2ae404f50` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:53.833194 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47dbcd6-2180-4093-9797-257706aceaa7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e200a1b99664fc8b41757e2ae404f50)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:53.836346 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47dbcd6-2180-4093-9797-257706aceaa7 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:53.888596 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-c47dbcd6-2180-4093-9797-257706aceaa7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e200a1b99664fc8b41757e2ae404f50', 'request_id': 'req-c47dbcd6-2180-4093-9797-257706aceaa7'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:53.894497 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-c47dbcd6-2180-4093-9797-257706aceaa7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e200a1b99664fc8b41757e2ae404f50', 'request_id': 'req-c47dbcd6-2180-4093-9797-257706aceaa7'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:53.901319 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-c47dbcd6-2180-4093-9797-257706aceaa7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e200a1b99664fc8b41757e2ae404f50', 'request_id': 'req-c47dbcd6-2180-4093-9797-257706aceaa7'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:53.913901 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-c47dbcd6-2180-4093-9797-257706aceaa7 admin admin] The token cache is being invalidate because project 3e200a1b99664fc8b41757e2ae404f50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:54:53.914189 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-c47dbcd6-2180-4093-9797-257706aceaa7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:53.914819 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-c47dbcd6-2180-4093-9797-257706aceaa7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:53.916365 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1357/2698] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:54:53 2022] DELETE /identity/v3/projects/3e200a1b99664fc8b41757e2ae404f50 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:54:53.922095 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4173f992-8f3e-47b5-a0c9-171cb42032a8 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:53.922833 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4173f992-8f3e-47b5-a0c9-171cb42032a8 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:53.923546 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4173f992-8f3e-47b5-a0c9-171cb42032a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:53.953981 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-4173f992-8f3e-47b5-a0c9-171cb42032a8 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:54:53.955225 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-4173f992-8f3e-47b5-a0c9-171cb42032a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:54.008368 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:54:54.020102 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1342/2699] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:54:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:54:54.028452 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e43cd7-7cc9-4efc-9d68-f1a05e9ccef2 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:54.028754 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e43cd7-7cc9-4efc-9d68-f1a05e9ccef2 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:54.029040 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e43cd7-7cc9-4efc-9d68-f1a05e9ccef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:54.047683 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-24e43cd7-7cc9-4efc-9d68-f1a05e9ccef2 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:54:54.048927 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-24e43cd7-7cc9-4efc-9d68-f1a05e9ccef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:54.057594 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1358/2700] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:54:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:54:54.061992 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4173f992-8f3e-47b5-a0c9-171cb42032a8 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:54.072059 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4173f992-8f3e-47b5-a0c9-171cb42032a8 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:54.076716 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c3fe1-84c7-4e42-bfcf-d4f52f8743aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:54.077421 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c3fe1-84c7-4e42-bfcf-d4f52f8743aa admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:54.077709 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c3fe1-84c7-4e42-bfcf-d4f52f8743aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:54.077991 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c3fe1-84c7-4e42-bfcf-d4f52f8743aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:54.078728 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77c3fe1-84c7-4e42-bfcf-d4f52f8743aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:54:54.079041 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77c3fe1-84c7-4e42-bfcf-d4f52f8743aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:54.080656 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77c3fe1-84c7-4e42-bfcf-d4f52f8743aa admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:54.095001 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1343/2701] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:54:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:54:54.099501 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e43cd7-7cc9-4efc-9d68-f1a05e9ccef2 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:54.109391 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e43cd7-7cc9-4efc-9d68-f1a05e9ccef2 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:54.114048 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889841e7-a5c1-48ee-8fab-bfe17b14e00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:54.114702 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889841e7-a5c1-48ee-8fab-bfe17b14e00d admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:54.115048 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889841e7-a5c1-48ee-8fab-bfe17b14e00d admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:54.116050 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889841e7-a5c1-48ee-8fab-bfe17b14e00d admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:54.116912 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889841e7-a5c1-48ee-8fab-bfe17b14e00d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:54.120793 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889841e7-a5c1-48ee-8fab-bfe17b14e00d admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:54.145196 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1359/2702] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:54:54 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:54:54.149701 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c3fe1-84c7-4e42-bfcf-d4f52f8743aa admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:54.161670 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c3fe1-84c7-4e42-bfcf-d4f52f8743aa admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:54.247870 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38197794-bb4e-41aa-8d54-57a0e0d3de89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:54.248838 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38197794-bb4e-41aa-8d54-57a0e0d3de89 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:54.249326 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38197794-bb4e-41aa-8d54-57a0e0d3de89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:54.249775 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38197794-bb4e-41aa-8d54-57a0e0d3de89 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:54.250633 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38197794-bb4e-41aa-8d54-57a0e0d3de89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:54.252765 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38197794-bb4e-41aa-8d54-57a0e0d3de89 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:54.292196 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1344/2703] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:54:54 2022] POST /identity/v3/users => generated 287 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:54:54.295194 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889841e7-a5c1-48ee-8fab-bfe17b14e00d admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:54.307186 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889841e7-a5c1-48ee-8fab-bfe17b14e00d admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:54.313781 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c775ad-fb11-4fee-a303-1f5130813e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:54.314554 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c775ad-fb11-4fee-a303-1f5130813e24 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:54.314886 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c775ad-fb11-4fee-a303-1f5130813e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:54.315143 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c775ad-fb11-4fee-a303-1f5130813e24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:54.316016 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c775ad-fb11-4fee-a303-1f5130813e24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:54.327009 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c775ad-fb11-4fee-a303-1f5130813e24 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:54.355674 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1360/2704] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:54:54 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:54:54.360536 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38197794-bb4e-41aa-8d54-57a0e0d3de89 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:54.373174 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38197794-bb4e-41aa-8d54-57a0e0d3de89 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:54.379435 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd5e8b9-4765-49e6-ad33-55141a1c4e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:54.383942 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5e8b9-4765-49e6-ad33-55141a1c4e43 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:54.384279 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5e8b9-4765-49e6-ad33-55141a1c4e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:54.384593 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5e8b9-4765-49e6-ad33-55141a1c4e43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:54.385364 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd5e8b9-4765-49e6-ad33-55141a1c4e43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:54.387043 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd5e8b9-4765-49e6-ad33-55141a1c4e43 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:54.397854 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1345/2705] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:54:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:54:54.402276 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c775ad-fb11-4fee-a303-1f5130813e24 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:54.414419 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c775ad-fb11-4fee-a303-1f5130813e24 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:54.419345 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0925db44-4532-4a2f-aa6c-6f10ef17a1e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:54.420656 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925db44-4532-4a2f-aa6c-6f10ef17a1e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:54.421027 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925db44-4532-4a2f-aa6c-6f10ef17a1e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:54.421408 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925db44-4532-4a2f-aa6c-6f10ef17a1e7 admin admin] PATH_INFO: `/v3/projects/8d922e774121418fa6517f9a41082991/users/0e8ad868bf304fdc807cba52abbd9049/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:54.453405 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0925db44-4532-4a2f-aa6c-6f10ef17a1e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d922e774121418fa6517f9a41082991, user_id=0e8ad868bf304fdc807cba52abbd9049, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:54.456120 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0925db44-4532-4a2f-aa6c-6f10ef17a1e7 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:54.470372 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1361/2706] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:54:54 2022] PUT /identity/v3/projects/8d922e774121418fa6517f9a41082991/users/0e8ad868bf304fdc807cba52abbd9049/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:54:54.481652 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025c61c-7885-4aee-9aff-d39c2d050cb5 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:54.482013 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025c61c-7885-4aee-9aff-d39c2d050cb5 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:54.482450 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025c61c-7885-4aee-9aff-d39c2d050cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:54.511415 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-5025c61c-7885-4aee-9aff-d39c2d050cb5 None None] MFA Rules not processed for user `0e8ad868bf304fdc807cba52abbd9049`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:54:54.512865 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-5025c61c-7885-4aee-9aff-d39c2d050cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:54.586296 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1346/2707] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:54:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:54:54.597397 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0925db44-4532-4a2f-aa6c-6f10ef17a1e7 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:54.605877 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-0925db44-4532-4a2f-aa6c-6f10ef17a1e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:54.629833 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:54:54.688846 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0925db44-4532-4a2f-aa6c-6f10ef17a1e7 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:54.698185 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132e79e6-2ff7-4914-88d8-b1a498a683ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:54.698889 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e79e6-2ff7-4914-88d8-b1a498a683ca service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:54.699177 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e79e6-2ff7-4914-88d8-b1a498a683ca service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:54.699455 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e79e6-2ff7-4914-88d8-b1a498a683ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:54.707919 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132e79e6-2ff7-4914-88d8-b1a498a683ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:54.709800 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132e79e6-2ff7-4914-88d8-b1a498a683ca service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:54.719362 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1362/2708] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:54:54 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:54:54.943153 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5025c61c-7885-4aee-9aff-d39c2d050cb5 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:55.006407 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:54:55.019789 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5025c61c-7885-4aee-9aff-d39c2d050cb5 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:55.025457 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cafc91-5acc-49c4-9d9f-31fd94a905fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:55.026654 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cafc91-5acc-49c4-9d9f-31fd94a905fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:55.027030 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cafc91-5acc-49c4-9d9f-31fd94a905fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:55.027323 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cafc91-5acc-49c4-9d9f-31fd94a905fe admin admin] PATH_INFO: `/v3/users/0e8ad868bf304fdc807cba52abbd9049` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:55.028838 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cafc91-5acc-49c4-9d9f-31fd94a905fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e8ad868bf304fdc807cba52abbd9049)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:55.032059 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cafc91-5acc-49c4-9d9f-31fd94a905fe admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:55.080957 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-62cafc91-5acc-49c4-9d9f-31fd94a905fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e8ad868bf304fdc807cba52abbd9049'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:55.081669 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-62cafc91-5acc-49c4-9d9f-31fd94a905fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e8ad868bf304fdc807cba52abbd9049'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:55.090483 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-62cafc91-5acc-49c4-9d9f-31fd94a905fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e8ad868bf304fdc807cba52abbd9049'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:55.102893 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-62cafc91-5acc-49c4-9d9f-31fd94a905fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e8ad868bf304fdc807cba52abbd9049'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:55.103550 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-62cafc91-5acc-49c4-9d9f-31fd94a905fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e8ad868bf304fdc807cba52abbd9049'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:55.117027 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1347/2709] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:54:54 2022] DELETE /identity/v3/users/0e8ad868bf304fdc807cba52abbd9049 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:54:55.122829 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215c74b-97ca-4bc4-af87-29c7239ba0ab None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:55.123370 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215c74b-97ca-4bc4-af87-29c7239ba0ab None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:55.123641 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215c74b-97ca-4bc4-af87-29c7239ba0ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:55.140881 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-9215c74b-97ca-4bc4-af87-29c7239ba0ab None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:54:55.142283 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-9215c74b-97ca-4bc4-af87-29c7239ba0ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:55.209689 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1363/2710] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:54:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:54:55.219244 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cafc91-5acc-49c4-9d9f-31fd94a905fe admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:55.220754 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-62cafc91-5acc-49c4-9d9f-31fd94a905fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:55.282635 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cafc91-5acc-49c4-9d9f-31fd94a905fe admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:55.286547 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c1ee23-6915-45c9-a147-5ebeb97b218f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:55.288667 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1ee23-6915-45c9-a147-5ebeb97b218f service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:55.288965 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1ee23-6915-45c9-a147-5ebeb97b218f service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:55.289303 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1ee23-6915-45c9-a147-5ebeb97b218f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:55.301225 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c1ee23-6915-45c9-a147-5ebeb97b218f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:55.301702 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c1ee23-6915-45c9-a147-5ebeb97b218f service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:55.312916 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1348/2711] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:54:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:54:55.839492 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215c74b-97ca-4bc4-af87-29c7239ba0ab None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:55.841074 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-9215c74b-97ca-4bc4-af87-29c7239ba0ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:55.857017 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215c74b-97ca-4bc4-af87-29c7239ba0ab None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:55.866541 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58a5cef-79aa-43cf-aa38-252d111e4d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:55.867448 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a5cef-79aa-43cf-aa38-252d111e4d38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:55.867758 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a5cef-79aa-43cf-aa38-252d111e4d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:55.868096 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a5cef-79aa-43cf-aa38-252d111e4d38 admin admin] PATH_INFO: `/v3/projects/8d922e774121418fa6517f9a41082991` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:55.869389 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58a5cef-79aa-43cf-aa38-252d111e4d38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d922e774121418fa6517f9a41082991)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:55.871848 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58a5cef-79aa-43cf-aa38-252d111e4d38 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:55.949674 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a58a5cef-79aa-43cf-aa38-252d111e4d38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d922e774121418fa6517f9a41082991', 'request_id': 'req-a58a5cef-79aa-43cf-aa38-252d111e4d38'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:55.955351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a58a5cef-79aa-43cf-aa38-252d111e4d38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d922e774121418fa6517f9a41082991', 'request_id': 'req-a58a5cef-79aa-43cf-aa38-252d111e4d38'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:55.962059 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a58a5cef-79aa-43cf-aa38-252d111e4d38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d922e774121418fa6517f9a41082991', 'request_id': 'req-a58a5cef-79aa-43cf-aa38-252d111e4d38'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:55.976380 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a58a5cef-79aa-43cf-aa38-252d111e4d38 admin admin] The token cache is being invalidate because project 8d922e774121418fa6517f9a41082991 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:54:55.976837 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a58a5cef-79aa-43cf-aa38-252d111e4d38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:55.977677 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a58a5cef-79aa-43cf-aa38-252d111e4d38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:54:55.979441 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1364/2712] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:54:55 2022] DELETE /identity/v3/projects/8d922e774121418fa6517f9a41082991 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:54:55.984280 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f8047-96fb-4623-8dfd-9c89f5b3c63a None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:55.984785 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f8047-96fb-4623-8dfd-9c89f5b3c63a None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:55.984882 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f8047-96fb-4623-8dfd-9c89f5b3c63a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:56.002903 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-690f8047-96fb-4623-8dfd-9c89f5b3c63a None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:54:56.004093 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-690f8047-96fb-4623-8dfd-9c89f5b3c63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:56.075000 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1349/2713] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:54:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:54:56.083808 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef4645-e91a-49da-87cb-049d06437f21 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:56.084177 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef4645-e91a-49da-87cb-049d06437f21 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:56.084460 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef4645-e91a-49da-87cb-049d06437f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:56.103900 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-b4ef4645-e91a-49da-87cb-049d06437f21 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:54:56.105117 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-b4ef4645-e91a-49da-87cb-049d06437f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:56.114096 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1365/2714] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:54:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:54:56.117981 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f8047-96fb-4623-8dfd-9c89f5b3c63a None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:56.129782 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f8047-96fb-4623-8dfd-9c89f5b3c63a None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:56.135023 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1000a70-dcf0-46a2-a7a9-b008177e9d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:56.135692 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1000a70-dcf0-46a2-a7a9-b008177e9d03 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:56.136006 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1000a70-dcf0-46a2-a7a9-b008177e9d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:56.136270 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1000a70-dcf0-46a2-a7a9-b008177e9d03 admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:56.137018 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1000a70-dcf0-46a2-a7a9-b008177e9d03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:54:56.137363 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1000a70-dcf0-46a2-a7a9-b008177e9d03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:56.138981 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1000a70-dcf0-46a2-a7a9-b008177e9d03 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:56.150740 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1350/2715] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:54:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:54:56.155860 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ef4645-e91a-49da-87cb-049d06437f21 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:56.165238 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ef4645-e91a-49da-87cb-049d06437f21 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:56.170278 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120b11b2-23e6-41a8-80b3-eea5b7e1f4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:56.170971 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b11b2-23e6-41a8-80b3-eea5b7e1f4c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:56.171265 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b11b2-23e6-41a8-80b3-eea5b7e1f4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:56.171550 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b11b2-23e6-41a8-80b3-eea5b7e1f4c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:56.172268 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120b11b2-23e6-41a8-80b3-eea5b7e1f4c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:56.174254 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120b11b2-23e6-41a8-80b3-eea5b7e1f4c4 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:56.201331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:54:56.209541 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1366/2716] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:54:56 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:54:56.213364 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1000a70-dcf0-46a2-a7a9-b008177e9d03 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:56.218681 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:54:56.224390 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1000a70-dcf0-46a2-a7a9-b008177e9d03 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:56.286189 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a7d8a-6092-4d05-948f-45f39750ab39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:56.286872 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a7d8a-6092-4d05-948f-45f39750ab39 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:56.287696 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a7d8a-6092-4d05-948f-45f39750ab39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:56.287976 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a7d8a-6092-4d05-948f-45f39750ab39 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:56.288669 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4a7d8a-6092-4d05-948f-45f39750ab39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:56.290637 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4a7d8a-6092-4d05-948f-45f39750ab39 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:56.312624 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1351/2717] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:54:56 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:54:56.316818 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120b11b2-23e6-41a8-80b3-eea5b7e1f4c4 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:56.333366 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120b11b2-23e6-41a8-80b3-eea5b7e1f4c4 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:56.348439 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70d1fe-706b-41a8-9cfa-b583bd769ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:56.349263 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70d1fe-706b-41a8-9cfa-b583bd769ab3 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:56.349657 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70d1fe-706b-41a8-9cfa-b583bd769ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:56.350033 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70d1fe-706b-41a8-9cfa-b583bd769ab3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:56.350683 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b70d1fe-706b-41a8-9cfa-b583bd769ab3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:56.352314 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b70d1fe-706b-41a8-9cfa-b583bd769ab3 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:56.363789 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1367/2718] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:54:56.367867 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a7d8a-6092-4d05-948f-45f39750ab39 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:56.379487 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a7d8a-6092-4d05-948f-45f39750ab39 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:56.385208 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ffba8b-083d-47f9-88f1-f6a127ab1375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:56.385833 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ffba8b-083d-47f9-88f1-f6a127ab1375 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:56.386118 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ffba8b-083d-47f9-88f1-f6a127ab1375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:56.386425 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ffba8b-083d-47f9-88f1-f6a127ab1375 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:56.387214 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ffba8b-083d-47f9-88f1-f6a127ab1375 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:56.388884 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ffba8b-083d-47f9-88f1-f6a127ab1375 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:56.400229 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1352/2719] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:54:56.404446 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70d1fe-706b-41a8-9cfa-b583bd769ab3 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:56.415216 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70d1fe-706b-41a8-9cfa-b583bd769ab3 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:56.420206 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1444b174-cf33-46ea-b284-5c21f7106f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:56.421089 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1444b174-cf33-46ea-b284-5c21f7106f61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:56.421409 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1444b174-cf33-46ea-b284-5c21f7106f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:56.421688 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1444b174-cf33-46ea-b284-5c21f7106f61 admin admin] PATH_INFO: `/v3/projects/d32636d2103a45d8ac1902c8321b4411/users/556a00b2cbce4d2ba929588ec3045eb4/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:56.436284 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1444b174-cf33-46ea-b284-5c21f7106f61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d32636d2103a45d8ac1902c8321b4411, user_id=556a00b2cbce4d2ba929588ec3045eb4, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:56.438945 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1444b174-cf33-46ea-b284-5c21f7106f61 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:56.453722 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1368/2720] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:54:56 2022] PUT /identity/v3/projects/d32636d2103a45d8ac1902c8321b4411/users/556a00b2cbce4d2ba929588ec3045eb4/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:54:56.464388 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ef4df-78c7-429d-9f7c-8945edb42473 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:56.464770 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ef4df-78c7-429d-9f7c-8945edb42473 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:56.465061 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ef4df-78c7-429d-9f7c-8945edb42473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:56.493173 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-d51ef4df-78c7-429d-9f7c-8945edb42473 None None] MFA Rules not processed for user `556a00b2cbce4d2ba929588ec3045eb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:54:56.494391 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-d51ef4df-78c7-429d-9f7c-8945edb42473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:56.561474 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1353/2721] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:54:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:54:56.566310 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1444b174-cf33-46ea-b284-5c21f7106f61 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:56.629602 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1444b174-cf33-46ea-b284-5c21f7106f61 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:56.634905 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e41978-2083-4393-9657-13a79513eeec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:56.635677 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e41978-2083-4393-9657-13a79513eeec admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:56.635953 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e41978-2083-4393-9657-13a79513eeec admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:56.636241 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e41978-2083-4393-9657-13a79513eeec admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:56.636966 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e41978-2083-4393-9657-13a79513eeec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:56.638956 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e41978-2083-4393-9657-13a79513eeec admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:56.666476 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1369/2722] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:54:56 2022] POST /identity/v3/projects => generated 367 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:54:56.670288 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51ef4df-78c7-429d-9f7c-8945edb42473 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:56.682157 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51ef4df-78c7-429d-9f7c-8945edb42473 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:56.739035 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06089e2f-0f67-4228-9db0-86b74ad1c87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:56.739717 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06089e2f-0f67-4228-9db0-86b74ad1c87d admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:56.740025 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06089e2f-0f67-4228-9db0-86b74ad1c87d admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:56.740818 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06089e2f-0f67-4228-9db0-86b74ad1c87d admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:56.741543 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06089e2f-0f67-4228-9db0-86b74ad1c87d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:56.743492 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06089e2f-0f67-4228-9db0-86b74ad1c87d admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:56.759403 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:54:56.764722 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1354/2723] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:54:56 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:54:56.768634 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e41978-2083-4393-9657-13a79513eeec admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:56.779269 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e41978-2083-4393-9657-13a79513eeec admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:56.785005 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42868b9-8bff-42ac-8b5b-d0e9403ef4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:56.786002 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42868b9-8bff-42ac-8b5b-d0e9403ef4c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:56.786414 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42868b9-8bff-42ac-8b5b-d0e9403ef4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:56.786792 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42868b9-8bff-42ac-8b5b-d0e9403ef4c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:56.787852 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42868b9-8bff-42ac-8b5b-d0e9403ef4c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:56.789960 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42868b9-8bff-42ac-8b5b-d0e9403ef4c8 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:56.801227 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1370/2724] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:54:56.806094 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06089e2f-0f67-4228-9db0-86b74ad1c87d admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:56.816247 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06089e2f-0f67-4228-9db0-86b74ad1c87d admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:56.821770 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f70c43-d062-40e2-856a-5f309d28fb5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:56.822677 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f70c43-d062-40e2-856a-5f309d28fb5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:56.822964 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f70c43-d062-40e2-856a-5f309d28fb5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:56.823248 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f70c43-d062-40e2-856a-5f309d28fb5b admin admin] PATH_INFO: `/v3/projects/fa08f503e4d345619cf6ecb35eca7ca4/users/09befbde89c84818ae570376bc228a8b/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:56.839173 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f70c43-d062-40e2-856a-5f309d28fb5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa08f503e4d345619cf6ecb35eca7ca4, user_id=09befbde89c84818ae570376bc228a8b, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:56.842176 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f70c43-d062-40e2-856a-5f309d28fb5b admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:56.856493 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1355/2725] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:54:56 2022] PUT /identity/v3/projects/fa08f503e4d345619cf6ecb35eca7ca4/users/09befbde89c84818ae570376bc228a8b/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:54:56.866068 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42868b9-8bff-42ac-8b5b-d0e9403ef4c8 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:56.877471 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42868b9-8bff-42ac-8b5b-d0e9403ef4c8 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:56.945527 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5519347-d492-4a45-a30a-025c04e574b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:56.946239 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5519347-d492-4a45-a30a-025c04e574b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:56.946528 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5519347-d492-4a45-a30a-025c04e574b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:56.946808 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5519347-d492-4a45-a30a-025c04e574b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:56.947545 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5519347-d492-4a45-a30a-025c04e574b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:56.949175 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5519347-d492-4a45-a30a-025c04e574b7 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:56.961183 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1371/2726] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:54:56.963895 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f70c43-d062-40e2-856a-5f309d28fb5b admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:56.975220 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f70c43-d062-40e2-856a-5f309d28fb5b admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:56.980018 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7720107-3236-46cd-9af4-af35373bd976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:56.980903 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7720107-3236-46cd-9af4-af35373bd976 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:56.981220 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7720107-3236-46cd-9af4-af35373bd976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:56.981802 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7720107-3236-46cd-9af4-af35373bd976 admin admin] PATH_INFO: `/v3/projects/fa08f503e4d345619cf6ecb35eca7ca4/users/09befbde89c84818ae570376bc228a8b/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:56.983752 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7720107-3236-46cd-9af4-af35373bd976 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa08f503e4d345619cf6ecb35eca7ca4, user_id=09befbde89c84818ae570376bc228a8b, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:56.986444 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7720107-3236-46cd-9af4-af35373bd976 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:56.998994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1356/2727] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:54:56 2022] PUT /identity/v3/projects/fa08f503e4d345619cf6ecb35eca7ca4/users/09befbde89c84818ae570376bc228a8b/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:54:57.009145 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5facff22-78a4-4e22-adc0-09b67a2babc1 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:57.009464 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5facff22-78a4-4e22-adc0-09b67a2babc1 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:57.009747 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5facff22-78a4-4e22-adc0-09b67a2babc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:57.038201 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-5facff22-78a4-4e22-adc0-09b67a2babc1 None None] MFA Rules not processed for user `09befbde89c84818ae570376bc228a8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:54:57.039407 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-5facff22-78a4-4e22-adc0-09b67a2babc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:57.064665 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:54:57.109684 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1372/2728] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:54:57 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:54:57.123312 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7720107-3236-46cd-9af4-af35373bd976 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:57.125059 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-c7720107-3236-46cd-9af4-af35373bd976 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:54:57.183865 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7720107-3236-46cd-9af4-af35373bd976 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:57.187471 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d04bc7-a5cb-4088-93aa-fdcf3305214e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:57.188125 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d04bc7-a5cb-4088-93aa-fdcf3305214e service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:57.188428 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d04bc7-a5cb-4088-93aa-fdcf3305214e service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:57.188695 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d04bc7-a5cb-4088-93aa-fdcf3305214e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:57.198811 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d04bc7-a5cb-4088-93aa-fdcf3305214e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:57.200236 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d04bc7-a5cb-4088-93aa-fdcf3305214e service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:57.211077 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1357/2729] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:54:57 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:54:57.282642 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5facff22-78a4-4e22-adc0-09b67a2babc1 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:54:57.292796 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5facff22-78a4-4e22-adc0-09b67a2babc1 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:54:57.296896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a733741-dfc7-41b8-a2a9-b0f8cf097fb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:54:57.297637 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a733741-dfc7-41b8-a2a9-b0f8cf097fb8 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:54:57.297930 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a733741-dfc7-41b8-a2a9-b0f8cf097fb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:54:57.298212 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a733741-dfc7-41b8-a2a9-b0f8cf097fb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:54:57.357141 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a733741-dfc7-41b8-a2a9-b0f8cf097fb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:54:57.358655 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a733741-dfc7-41b8-a2a9-b0f8cf097fb8 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:54:57.368121 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1373/2730] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:54:57 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:55:07.205693 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=41,keystone:INSERT=2 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:55:07.363538 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=46 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:55:18.177926 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d04bc7-a5cb-4088-93aa-fdcf3305214e service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:55:18.179449 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-34d04bc7-a5cb-4088-93aa-fdcf3305214e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:55:18.252215 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d04bc7-a5cb-4088-93aa-fdcf3305214e service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:55:18.256756 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4037a2c0-17c1-496a-8eca-b19b56e1b54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:55:18.257666 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4037a2c0-17c1-496a-8eca-b19b56e1b54a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:18.257956 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4037a2c0-17c1-496a-8eca-b19b56e1b54a admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:18.258238 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4037a2c0-17c1-496a-8eca-b19b56e1b54a admin admin] PATH_INFO: `/v3/users/d78b366335cd47a6a9ff44779496f7ea` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:18.259447 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4037a2c0-17c1-496a-8eca-b19b56e1b54a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d78b366335cd47a6a9ff44779496f7ea)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:55:18.261452 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4037a2c0-17c1-496a-8eca-b19b56e1b54a admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:55:18.308002 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-4037a2c0-17c1-496a-8eca-b19b56e1b54a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd78b366335cd47a6a9ff44779496f7ea'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:18.308644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-4037a2c0-17c1-496a-8eca-b19b56e1b54a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd78b366335cd47a6a9ff44779496f7ea'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:18.315740 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-4037a2c0-17c1-496a-8eca-b19b56e1b54a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd78b366335cd47a6a9ff44779496f7ea'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:18.327778 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-4037a2c0-17c1-496a-8eca-b19b56e1b54a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd78b366335cd47a6a9ff44779496f7ea'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:18.328423 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-4037a2c0-17c1-496a-8eca-b19b56e1b54a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd78b366335cd47a6a9ff44779496f7ea'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:18.340799 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1358/2731] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:55:18 2022] DELETE /identity/v3/users/d78b366335cd47a6a9ff44779496f7ea => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:55:18.345450 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a34f9-aa0e-4b46-bb0b-e3895b8e3730 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:18.345936 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a34f9-aa0e-4b46-bb0b-e3895b8e3730 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:18.346166 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a34f9-aa0e-4b46-bb0b-e3895b8e3730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:18.363678 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-2e7a34f9-aa0e-4b46-bb0b-e3895b8e3730 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:55:18.364883 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-2e7a34f9-aa0e-4b46-bb0b-e3895b8e3730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:55:18.423496 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1374/2732] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:55:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:55:18.433085 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4037a2c0-17c1-496a-8eca-b19b56e1b54a admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:55:18.434590 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-4037a2c0-17c1-496a-8eca-b19b56e1b54a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:55:18.493401 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4037a2c0-17c1-496a-8eca-b19b56e1b54a admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:55:18.497346 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3720af-39c8-43b7-bbed-3a133f885dac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:55:18.497997 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3720af-39c8-43b7-bbed-3a133f885dac service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:18.498283 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3720af-39c8-43b7-bbed-3a133f885dac service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:18.498563 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3720af-39c8-43b7-bbed-3a133f885dac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:18.508040 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3720af-39c8-43b7-bbed-3a133f885dac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:55:18.509517 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3720af-39c8-43b7-bbed-3a133f885dac service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:55:18.518750 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1359/2733] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:55:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:55:18.790675 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7a34f9-aa0e-4b46-bb0b-e3895b8e3730 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:55:18.792815 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-2e7a34f9-aa0e-4b46-bb0b-e3895b8e3730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:55:18.810016 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7a34f9-aa0e-4b46-bb0b-e3895b8e3730 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:55:18.815191 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eb415b-7415-413e-8d62-637b289317f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:55:18.816075 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb415b-7415-413e-8d62-637b289317f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:18.816370 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb415b-7415-413e-8d62-637b289317f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:18.816657 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb415b-7415-413e-8d62-637b289317f7 admin admin] PATH_INFO: `/v3/projects/da1eaa88760a4f3bb9d2b90280b07ca2` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:18.817900 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eb415b-7415-413e-8d62-637b289317f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da1eaa88760a4f3bb9d2b90280b07ca2)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:55:18.820397 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eb415b-7415-413e-8d62-637b289317f7 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:55:18.869711 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-33eb415b-7415-413e-8d62-637b289317f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da1eaa88760a4f3bb9d2b90280b07ca2', 'request_id': 'req-33eb415b-7415-413e-8d62-637b289317f7'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:18.875608 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-33eb415b-7415-413e-8d62-637b289317f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da1eaa88760a4f3bb9d2b90280b07ca2', 'request_id': 'req-33eb415b-7415-413e-8d62-637b289317f7'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:18.881671 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-33eb415b-7415-413e-8d62-637b289317f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da1eaa88760a4f3bb9d2b90280b07ca2', 'request_id': 'req-33eb415b-7415-413e-8d62-637b289317f7'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:18.894602 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-33eb415b-7415-413e-8d62-637b289317f7 admin admin] The token cache is being invalidate because project da1eaa88760a4f3bb9d2b90280b07ca2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:55:18.894893 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-33eb415b-7415-413e-8d62-637b289317f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:18.895561 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-33eb415b-7415-413e-8d62-637b289317f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:18.897093 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1375/2734] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:55:18 2022] DELETE /identity/v3/projects/da1eaa88760a4f3bb9d2b90280b07ca2 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:55:18.901964 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a765a6-900b-432d-8d8a-1d947fc39158 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:18.902255 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a765a6-900b-432d-8d8a-1d947fc39158 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:18.902535 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a765a6-900b-432d-8d8a-1d947fc39158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:18.920497 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-f7a765a6-900b-432d-8d8a-1d947fc39158 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:55:18.921737 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-f7a765a6-900b-432d-8d8a-1d947fc39158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:55:18.939041 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:55:18.982280 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1360/2735] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:55:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:55:18.989896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb19762-c048-4d28-a824-f932e9097d6b None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:18.990193 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb19762-c048-4d28-a824-f932e9097d6b None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:18.990478 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb19762-c048-4d28-a824-f932e9097d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:19.009686 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-9eb19762-c048-4d28-a824-f932e9097d6b None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:55:19.010888 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-9eb19762-c048-4d28-a824-f932e9097d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:55:19.018785 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1376/2736] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:55:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:55:19.023296 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a765a6-900b-432d-8d8a-1d947fc39158 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:55:19.032768 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a765a6-900b-432d-8d8a-1d947fc39158 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:55:19.037382 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e964fa7e-3975-4e37-a9fd-a9e274e6414c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:55:19.038036 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e964fa7e-3975-4e37-a9fd-a9e274e6414c admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:19.038890 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e964fa7e-3975-4e37-a9fd-a9e274e6414c admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:19.039169 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e964fa7e-3975-4e37-a9fd-a9e274e6414c admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:19.039898 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e964fa7e-3975-4e37-a9fd-a9e274e6414c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:55:19.040211 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e964fa7e-3975-4e37-a9fd-a9e274e6414c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:55:19.041860 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e964fa7e-3975-4e37-a9fd-a9e274e6414c admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:55:19.053859 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1361/2737] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:55:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:55:19.057735 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb19762-c048-4d28-a824-f932e9097d6b None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:55:19.066603 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb19762-c048-4d28-a824-f932e9097d6b None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:55:19.072434 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca9088e-7bb9-4f01-a6d1-ea7eaec45013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:55:19.073182 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9088e-7bb9-4f01-a6d1-ea7eaec45013 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:19.073487 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9088e-7bb9-4f01-a6d1-ea7eaec45013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:19.073769 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9088e-7bb9-4f01-a6d1-ea7eaec45013 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:19.074480 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca9088e-7bb9-4f01-a6d1-ea7eaec45013 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:55:19.076432 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca9088e-7bb9-4f01-a6d1-ea7eaec45013 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:55:19.111013 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1377/2738] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:55:19 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:55:19.116045 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e964fa7e-3975-4e37-a9fd-a9e274e6414c admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:55:19.126720 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e964fa7e-3975-4e37-a9fd-a9e274e6414c admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:55:19.183995 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e5cf59-07ae-4117-82e2-96f50793b862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:55:19.184685 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e5cf59-07ae-4117-82e2-96f50793b862 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:19.184980 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e5cf59-07ae-4117-82e2-96f50793b862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:19.185295 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e5cf59-07ae-4117-82e2-96f50793b862 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:19.186008 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e5cf59-07ae-4117-82e2-96f50793b862 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:55:19.187924 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e5cf59-07ae-4117-82e2-96f50793b862 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:55:19.210523 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1362/2739] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:55:19 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:55:19.214460 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca9088e-7bb9-4f01-a6d1-ea7eaec45013 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:55:19.224537 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca9088e-7bb9-4f01-a6d1-ea7eaec45013 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:55:19.231281 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3054363-6cf5-4bf4-b3f9-3e77b9926ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:55:19.232098 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3054363-6cf5-4bf4-b3f9-3e77b9926ac8 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:19.232525 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3054363-6cf5-4bf4-b3f9-3e77b9926ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:19.232826 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3054363-6cf5-4bf4-b3f9-3e77b9926ac8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:19.233621 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3054363-6cf5-4bf4-b3f9-3e77b9926ac8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:55:19.235257 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3054363-6cf5-4bf4-b3f9-3e77b9926ac8 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:55:19.248478 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1378/2740] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:55:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:55:19.252482 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e5cf59-07ae-4117-82e2-96f50793b862 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:55:19.262490 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e5cf59-07ae-4117-82e2-96f50793b862 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:55:19.267358 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92b1bac-004f-4664-a69b-156f7430a238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:55:19.268032 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b1bac-004f-4664-a69b-156f7430a238 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:19.268326 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b1bac-004f-4664-a69b-156f7430a238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:19.268615 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b1bac-004f-4664-a69b-156f7430a238 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:19.269463 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92b1bac-004f-4664-a69b-156f7430a238 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:55:19.271256 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92b1bac-004f-4664-a69b-156f7430a238 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:55:19.283257 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1363/2741] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:55:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:55:19.287451 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3054363-6cf5-4bf4-b3f9-3e77b9926ac8 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:55:19.297953 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3054363-6cf5-4bf4-b3f9-3e77b9926ac8 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:55:19.302617 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd4a494-c2e8-4a89-9ba7-a4863b08a7c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:55:19.303489 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4a494-c2e8-4a89-9ba7-a4863b08a7c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:19.303838 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4a494-c2e8-4a89-9ba7-a4863b08a7c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:19.304126 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4a494-c2e8-4a89-9ba7-a4863b08a7c9 admin admin] PATH_INFO: `/v3/projects/a5255dd29f1d4683a6a03eaf63def876/users/56a9c90a37c94b81a167e6a8e9fd9f91/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:19.318295 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd4a494-c2e8-4a89-9ba7-a4863b08a7c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5255dd29f1d4683a6a03eaf63def876, user_id=56a9c90a37c94b81a167e6a8e9fd9f91, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:55:19.320979 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd4a494-c2e8-4a89-9ba7-a4863b08a7c9 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:55:19.333319 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1379/2742] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:55:19 2022] PUT /identity/v3/projects/a5255dd29f1d4683a6a03eaf63def876/users/56a9c90a37c94b81a167e6a8e9fd9f91/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:55:19.344466 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d9022-867a-4fc8-a57b-f412c7f1c54c None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:19.344762 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d9022-867a-4fc8-a57b-f412c7f1c54c None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:19.345045 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d9022-867a-4fc8-a57b-f412c7f1c54c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:19.371499 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-018d9022-867a-4fc8-a57b-f412c7f1c54c None None] MFA Rules not processed for user `56a9c90a37c94b81a167e6a8e9fd9f91`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:55:19.372704 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-018d9022-867a-4fc8-a57b-f412c7f1c54c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:55:19.436792 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1364/2743] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:55:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:55:19.448191 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd4a494-c2e8-4a89-9ba7-a4863b08a7c9 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:55:19.449903 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-6bd4a494-c2e8-4a89-9ba7-a4863b08a7c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:55:19.473127 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:55:19.528562 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd4a494-c2e8-4a89-9ba7-a4863b08a7c9 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:55:19.532126 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc5307-7409-4416-9755-a89187d5690d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:55:19.532763 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc5307-7409-4416-9755-a89187d5690d service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:19.533047 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc5307-7409-4416-9755-a89187d5690d service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:19.533355 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc5307-7409-4416-9755-a89187d5690d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:19.542343 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dc5307-7409-4416-9755-a89187d5690d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:55:19.543780 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dc5307-7409-4416-9755-a89187d5690d service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:55:19.552464 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1380/2744] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:55:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:55:22.929827 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac8088-96d0-4b04-8186-6ec1fcdb768e None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:22.930347 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac8088-96d0-4b04-8186-6ec1fcdb768e None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:22.930478 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac8088-96d0-4b04-8186-6ec1fcdb768e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:22.947929 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-feac8088-96d0-4b04-8186-6ec1fcdb768e None None] MFA Rules not processed for user `a30dc1dd2b00463bb3c7957864c56043`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:55:22.949117 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-feac8088-96d0-4b04-8186-6ec1fcdb768e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:55:22.960905 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:55:23.005939 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1365/2745] 10.222.0.36 () {58 vars in 1009 bytes} [Thu May 19 16:55:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:55:23.056469 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc5307-7409-4416-9755-a89187d5690d service cinder] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:55:23.058251 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-21dc5307-7409-4416-9755-a89187d5690d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:55:23.144345 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc5307-7409-4416-9755-a89187d5690d service cinder] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:55:23.151797 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1268dc6-6e19-471c-a957-354fdd7a1098 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffe7cebc6f3b40d8a8e1866ed2da9822', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:55:23.153351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1268dc6-6e19-471c-a957-354fdd7a1098 service swift] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:23.153646 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1268dc6-6e19-471c-a957-354fdd7a1098 service swift] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:23.154304 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1268dc6-6e19-471c-a957-354fdd7a1098 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:23.166486 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1268dc6-6e19-471c-a957-354fdd7a1098 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:55:23.169427 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1268dc6-6e19-471c-a957-354fdd7a1098 service swift] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:55:23.193597 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1381/2746] 10.222.0.36 () {60 vars in 1384 bytes} [Thu May 19 16:55:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 16:55:32.999247 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=17 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:55:33.177690 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=58 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:55:36.349386 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac8088-96d0-4b04-8186-6ec1fcdb768e None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:55:36.426072 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac8088-96d0-4b04-8186-6ec1fcdb768e None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:55:36.432425 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ebf886-ca45-4235-ac8a-9cc3eb8c866d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:55:36.433354 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ebf886-ca45-4235-ac8a-9cc3eb8c866d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:36.433645 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ebf886-ca45-4235-ac8a-9cc3eb8c866d admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:36.433929 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ebf886-ca45-4235-ac8a-9cc3eb8c866d admin admin] PATH_INFO: `/v3/users/56a9c90a37c94b81a167e6a8e9fd9f91` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:36.435158 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ebf886-ca45-4235-ac8a-9cc3eb8c866d admin admin] RBAC: Authorizing `identity:delete_user(user_id=56a9c90a37c94b81a167e6a8e9fd9f91)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:55:36.437160 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ebf886-ca45-4235-ac8a-9cc3eb8c866d admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:55:36.526071 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f4ebf886-ca45-4235-ac8a-9cc3eb8c866d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56a9c90a37c94b81a167e6a8e9fd9f91'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:36.526779 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f4ebf886-ca45-4235-ac8a-9cc3eb8c866d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56a9c90a37c94b81a167e6a8e9fd9f91'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:36.537203 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f4ebf886-ca45-4235-ac8a-9cc3eb8c866d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56a9c90a37c94b81a167e6a8e9fd9f91'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:36.554219 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f4ebf886-ca45-4235-ac8a-9cc3eb8c866d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56a9c90a37c94b81a167e6a8e9fd9f91'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:36.555363 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-f4ebf886-ca45-4235-ac8a-9cc3eb8c866d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56a9c90a37c94b81a167e6a8e9fd9f91'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:36.580178 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1366/2747] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:55:36 2022] DELETE /identity/v3/users/56a9c90a37c94b81a167e6a8e9fd9f91 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:55:36.581788 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516219f-8ac5-4d1e-a9dc-725744deeebb None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:36.582360 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516219f-8ac5-4d1e-a9dc-725744deeebb None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:36.582360 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516219f-8ac5-4d1e-a9dc-725744deeebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:36.601676 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-1516219f-8ac5-4d1e-a9dc-725744deeebb None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:55:36.603201 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-1516219f-8ac5-4d1e-a9dc-725744deeebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:55:36.675866 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1382/2748] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:55:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:55:36.686173 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ebf886-ca45-4235-ac8a-9cc3eb8c866d admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:55:36.687634 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-f4ebf886-ca45-4235-ac8a-9cc3eb8c866d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:55:36.765007 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ebf886-ca45-4235-ac8a-9cc3eb8c866d admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:55:36.768825 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0b245d-f3ae-41a9-b456-34c6451562a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:55:36.769603 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b245d-f3ae-41a9-b456-34c6451562a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:36.769968 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b245d-f3ae-41a9-b456-34c6451562a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:36.770319 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b245d-f3ae-41a9-b456-34c6451562a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:36.781477 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0b245d-f3ae-41a9-b456-34c6451562a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:55:36.783055 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0b245d-f3ae-41a9-b456-34c6451562a3 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:55:36.795625 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1367/2749] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:55:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:55:37.125864 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1516219f-8ac5-4d1e-a9dc-725744deeebb None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:55:37.127623 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-1516219f-8ac5-4d1e-a9dc-725744deeebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:55:37.144965 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1516219f-8ac5-4d1e-a9dc-725744deeebb None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:55:37.152228 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70db3f3-1e17-43f0-b947-8c09fcb8ce64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:55:37.154682 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70db3f3-1e17-43f0-b947-8c09fcb8ce64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:37.155427 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70db3f3-1e17-43f0-b947-8c09fcb8ce64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:37.156181 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70db3f3-1e17-43f0-b947-8c09fcb8ce64 admin admin] PATH_INFO: `/v3/projects/a5255dd29f1d4683a6a03eaf63def876` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:37.158198 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70db3f3-1e17-43f0-b947-8c09fcb8ce64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5255dd29f1d4683a6a03eaf63def876)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:55:37.163629 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70db3f3-1e17-43f0-b947-8c09fcb8ce64 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:55:37.256390 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-c70db3f3-1e17-43f0-b947-8c09fcb8ce64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5255dd29f1d4683a6a03eaf63def876', 'request_id': 'req-c70db3f3-1e17-43f0-b947-8c09fcb8ce64'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:37.263768 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-c70db3f3-1e17-43f0-b947-8c09fcb8ce64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5255dd29f1d4683a6a03eaf63def876', 'request_id': 'req-c70db3f3-1e17-43f0-b947-8c09fcb8ce64'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:37.271002 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-c70db3f3-1e17-43f0-b947-8c09fcb8ce64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5255dd29f1d4683a6a03eaf63def876', 'request_id': 'req-c70db3f3-1e17-43f0-b947-8c09fcb8ce64'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:37.293973 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-c70db3f3-1e17-43f0-b947-8c09fcb8ce64 admin admin] The token cache is being invalidate because project a5255dd29f1d4683a6a03eaf63def876 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:55:37.294381 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-c70db3f3-1e17-43f0-b947-8c09fcb8ce64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:37.295094 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-c70db3f3-1e17-43f0-b947-8c09fcb8ce64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:37.297853 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1383/2750] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:55:37 2022] DELETE /identity/v3/projects/a5255dd29f1d4683a6a03eaf63def876 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:55:45.922413 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59adc2-362b-4897-958f-e9af401228cd None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:45.922865 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59adc2-362b-4897-958f-e9af401228cd None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:45.922966 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59adc2-362b-4897-958f-e9af401228cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:45.954818 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:55:45.984548 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-5f59adc2-362b-4897-958f-e9af401228cd None None] MFA Rules not processed for user `a30dc1dd2b00463bb3c7957864c56043`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:55:45.985780 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-5f59adc2-362b-4897-958f-e9af401228cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:55:46.054535 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1368/2751] 10.222.0.36 () {58 vars in 967 bytes} [Thu May 19 16:55:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:55:46.066755 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70db3f3-1e17-43f0-b947-8c09fcb8ce64 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:55:46.068121 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-c70db3f3-1e17-43f0-b947-8c09fcb8ce64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:55:46.175251 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70db3f3-1e17-43f0-b947-8c09fcb8ce64 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:55:46.182872 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20e9ac1-a700-44bc-bf0b-6d346f77f4b3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffe7cebc6f3b40d8a8e1866ed2da9822', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:55:46.183621 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e9ac1-a700-44bc-bf0b-6d346f77f4b3 service swift] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:46.184005 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e9ac1-a700-44bc-bf0b-6d346f77f4b3 service swift] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:46.184314 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e9ac1-a700-44bc-bf0b-6d346f77f4b3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:46.190998 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:55:46.196407 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20e9ac1-a700-44bc-bf0b-6d346f77f4b3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:55:46.198851 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20e9ac1-a700-44bc-bf0b-6d346f77f4b3 service swift] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:55:46.211746 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1384/2752] 10.222.0.36 () {60 vars in 1384 bytes} [Thu May 19 16:55:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 16:55:47.661413 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f59adc2-362b-4897-958f-e9af401228cd None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:55:47.662955 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-5f59adc2-362b-4897-958f-e9af401228cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:55:47.722651 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f59adc2-362b-4897-958f-e9af401228cd None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:55:47.726808 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6039b-912c-477a-b720-f3ed08a2b328 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:55:47.728016 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6039b-912c-477a-b720-f3ed08a2b328 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:47.728304 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6039b-912c-477a-b720-f3ed08a2b328 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:47.728584 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6039b-912c-477a-b720-f3ed08a2b328 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:47.730445 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-88d6039b-912c-477a-b720-f3ed08a2b328 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:55:47.740390 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d6039b-912c-477a-b720-f3ed08a2b328 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:55:47.741898 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d6039b-912c-477a-b720-f3ed08a2b328 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:55:47.751771 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1369/2753] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:55:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:55:47.839230 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20e9ac1-a700-44bc-bf0b-6d346f77f4b3 service swift] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:55:47.848845 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20e9ac1-a700-44bc-bf0b-6d346f77f4b3 service swift] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:55:47.853043 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff48da6-aa54-4580-8fe6-a3a86fb39cba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:55:47.853718 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff48da6-aa54-4580-8fe6-a3a86fb39cba service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:47.854005 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff48da6-aa54-4580-8fe6-a3a86fb39cba service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:47.854284 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff48da6-aa54-4580-8fe6-a3a86fb39cba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:47.856129 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-4ff48da6-aa54-4580-8fe6-a3a86fb39cba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:55:47.936577 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff48da6-aa54-4580-8fe6-a3a86fb39cba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:55:47.938084 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff48da6-aa54-4580-8fe6-a3a86fb39cba service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:55:47.950402 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1385/2754] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:55:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:55:48.578356 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6039b-912c-477a-b720-f3ed08a2b328 service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:55:48.579661 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff48da6-aa54-4580-8fe6-a3a86fb39cba service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:55:48.581173 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-88d6039b-912c-477a-b720-f3ed08a2b328 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:55:48.582978 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-4ff48da6-aa54-4580-8fe6-a3a86fb39cba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:55:48.661100 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6039b-912c-477a-b720-f3ed08a2b328 service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:55:48.665489 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff48da6-aa54-4580-8fe6-a3a86fb39cba service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:55:48.666890 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1eabe2-e09f-4231-a3f5-5a4f72d266de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:55:48.667806 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1eabe2-e09f-4231-a3f5-5a4f72d266de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:48.668092 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1eabe2-e09f-4231-a3f5-5a4f72d266de admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:48.668573 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1eabe2-e09f-4231-a3f5-5a4f72d266de admin admin] PATH_INFO: `/v3/users/556a00b2cbce4d2ba929588ec3045eb4` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:48.670038 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1eabe2-e09f-4231-a3f5-5a4f72d266de admin admin] RBAC: Authorizing `identity:delete_user(user_id=556a00b2cbce4d2ba929588ec3045eb4)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:55:48.671086 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed79f148-0600-4de3-a8be-3d1032e1c1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:55:48.671964 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1eabe2-e09f-4231-a3f5-5a4f72d266de admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:55:48.672123 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79f148-0600-4de3-a8be-3d1032e1c1f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:48.672301 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79f148-0600-4de3-a8be-3d1032e1c1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:48.672527 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79f148-0600-4de3-a8be-3d1032e1c1f5 admin admin] PATH_INFO: `/v3/users/8d0d2a7bb14c44a3b28a11274a3a351e` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:48.673796 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed79f148-0600-4de3-a8be-3d1032e1c1f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d0d2a7bb14c44a3b28a11274a3a351e)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:55:48.675800 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed79f148-0600-4de3-a8be-3d1032e1c1f5 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:55:48.699142 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:55:48.726139 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8e1eabe2-e09f-4231-a3f5-5a4f72d266de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '556a00b2cbce4d2ba929588ec3045eb4'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:48.727582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8e1eabe2-e09f-4231-a3f5-5a4f72d266de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '556a00b2cbce4d2ba929588ec3045eb4'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:48.739314 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8e1eabe2-e09f-4231-a3f5-5a4f72d266de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '556a00b2cbce4d2ba929588ec3045eb4'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:48.741326 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ed79f148-0600-4de3-a8be-3d1032e1c1f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d0d2a7bb14c44a3b28a11274a3a351e'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:48.742225 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ed79f148-0600-4de3-a8be-3d1032e1c1f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d0d2a7bb14c44a3b28a11274a3a351e'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:48.743465 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ed79f148-0600-4de3-a8be-3d1032e1c1f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d0d2a7bb14c44a3b28a11274a3a351e'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:48.753808 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ed79f148-0600-4de3-a8be-3d1032e1c1f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d0d2a7bb14c44a3b28a11274a3a351e'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:48.760605 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8e1eabe2-e09f-4231-a3f5-5a4f72d266de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '556a00b2cbce4d2ba929588ec3045eb4'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:48.761340 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-8e1eabe2-e09f-4231-a3f5-5a4f72d266de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '556a00b2cbce4d2ba929588ec3045eb4'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:48.769382 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ed79f148-0600-4de3-a8be-3d1032e1c1f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d0d2a7bb14c44a3b28a11274a3a351e'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:48.775843 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1370/2755] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:55:48 2022] DELETE /identity/v3/users/556a00b2cbce4d2ba929588ec3045eb4 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:55:48.780872 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac177e-2ac0-44b1-a1b7-ca2e603bfa71 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:48.781398 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac177e-2ac0-44b1-a1b7-ca2e603bfa71 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:48.781876 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac177e-2ac0-44b1-a1b7-ca2e603bfa71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:48.783232 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:55:48.791387 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1386/2756] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:55:48 2022] DELETE /identity/v3/users/8d0d2a7bb14c44a3b28a11274a3a351e => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:55:48.814930 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-46ac177e-2ac0-44b1-a1b7-ca2e603bfa71 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:55:48.816592 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-46ac177e-2ac0-44b1-a1b7-ca2e603bfa71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:55:48.892826 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1371/2757] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:55:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:55:48.902595 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed79f148-0600-4de3-a8be-3d1032e1c1f5 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:55:48.904698 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-ed79f148-0600-4de3-a8be-3d1032e1c1f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:55:48.975698 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed79f148-0600-4de3-a8be-3d1032e1c1f5 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:55:48.980238 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ef37e-48b0-4203-93d7-927d82923027 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:55:48.980898 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ef37e-48b0-4203-93d7-927d82923027 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:48.981221 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ef37e-48b0-4203-93d7-927d82923027 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:48.981592 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ef37e-48b0-4203-93d7-927d82923027 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:48.991408 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ef37e-48b0-4203-93d7-927d82923027 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:55:48.992968 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ef37e-48b0-4203-93d7-927d82923027 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:55:49.003689 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1387/2758] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:55:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:55:49.069566 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac177e-2ac0-44b1-a1b7-ca2e603bfa71 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:55:49.073419 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-46ac177e-2ac0-44b1-a1b7-ca2e603bfa71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:55:49.092137 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac177e-2ac0-44b1-a1b7-ca2e603bfa71 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:55:49.099850 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae221d43-2998-4f7a-9e0c-171bdcbb048f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:55:49.100709 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae221d43-2998-4f7a-9e0c-171bdcbb048f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:49.100997 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae221d43-2998-4f7a-9e0c-171bdcbb048f admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:49.101458 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae221d43-2998-4f7a-9e0c-171bdcbb048f admin admin] PATH_INFO: `/v3/projects/efe37371967543a3aa0834579ad5e714` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:49.115627 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae221d43-2998-4f7a-9e0c-171bdcbb048f admin admin] RBAC: Authorizing `identity:delete_project(project_id=efe37371967543a3aa0834579ad5e714)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:55:49.117755 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae221d43-2998-4f7a-9e0c-171bdcbb048f admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:55:49.201584 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-ae221d43-2998-4f7a-9e0c-171bdcbb048f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efe37371967543a3aa0834579ad5e714', 'request_id': 'req-ae221d43-2998-4f7a-9e0c-171bdcbb048f'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:49.209543 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-ae221d43-2998-4f7a-9e0c-171bdcbb048f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efe37371967543a3aa0834579ad5e714', 'request_id': 'req-ae221d43-2998-4f7a-9e0c-171bdcbb048f'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:49.216938 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-ae221d43-2998-4f7a-9e0c-171bdcbb048f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efe37371967543a3aa0834579ad5e714', 'request_id': 'req-ae221d43-2998-4f7a-9e0c-171bdcbb048f'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:49.233351 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-ae221d43-2998-4f7a-9e0c-171bdcbb048f admin admin] The token cache is being invalidate because project efe37371967543a3aa0834579ad5e714 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:55:49.233738 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-ae221d43-2998-4f7a-9e0c-171bdcbb048f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:49.234295 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-ae221d43-2998-4f7a-9e0c-171bdcbb048f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:49.236219 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1372/2759] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:55:49 2022] DELETE /identity/v3/projects/efe37371967543a3aa0834579ad5e714 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:55:49.241608 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a3a48-8bd1-42b0-9d67-3e21c52e5829 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:49.242088 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a3a48-8bd1-42b0-9d67-3e21c52e5829 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:49.242711 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a3a48-8bd1-42b0-9d67-3e21c52e5829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:49.263670 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-9c5a3a48-8bd1-42b0-9d67-3e21c52e5829 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:55:49.264909 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-9c5a3a48-8bd1-42b0-9d67-3e21c52e5829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:55:49.331468 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae221d43-2998-4f7a-9e0c-171bdcbb048f admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:55:49.334001 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ae221d43-2998-4f7a-9e0c-171bdcbb048f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:55:49.345185 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1388/2760] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:55:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:55:49.349479 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae221d43-2998-4f7a-9e0c-171bdcbb048f admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:55:49.354419 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63fd1c-32fe-4543-a4ad-15cf030900fa None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:49.354827 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63fd1c-32fe-4543-a4ad-15cf030900fa None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:49.355223 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63fd1c-32fe-4543-a4ad-15cf030900fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:49.356436 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2852d-cc5e-4a16-9b65-f126372b216e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:55:49.357346 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2852d-cc5e-4a16-9b65-f126372b216e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:49.357639 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2852d-cc5e-4a16-9b65-f126372b216e admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:49.357922 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2852d-cc5e-4a16-9b65-f126372b216e admin admin] PATH_INFO: `/v3/projects/d32636d2103a45d8ac1902c8321b4411` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:49.359189 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd2852d-cc5e-4a16-9b65-f126372b216e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d32636d2103a45d8ac1902c8321b4411)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:55:49.361192 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd2852d-cc5e-4a16-9b65-f126372b216e admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:55:49.380676 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-9e63fd1c-32fe-4543-a4ad-15cf030900fa None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:55:49.382008 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-9e63fd1c-32fe-4543-a4ad-15cf030900fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:55:49.388972 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:55:49.394462 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1389/2761] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:55:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:55:49.395738 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e63fd1c-32fe-4543-a4ad-15cf030900fa None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:55:49.412656 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e63fd1c-32fe-4543-a4ad-15cf030900fa None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:55:49.423739 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-2bd2852d-cc5e-4a16-9b65-f126372b216e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd32636d2103a45d8ac1902c8321b4411', 'request_id': 'req-2bd2852d-cc5e-4a16-9b65-f126372b216e'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:49.430566 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-2bd2852d-cc5e-4a16-9b65-f126372b216e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd32636d2103a45d8ac1902c8321b4411', 'request_id': 'req-2bd2852d-cc5e-4a16-9b65-f126372b216e'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:49.437641 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-2bd2852d-cc5e-4a16-9b65-f126372b216e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd32636d2103a45d8ac1902c8321b4411', 'request_id': 'req-2bd2852d-cc5e-4a16-9b65-f126372b216e'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:49.450920 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-2bd2852d-cc5e-4a16-9b65-f126372b216e admin admin] The token cache is being invalidate because project d32636d2103a45d8ac1902c8321b4411 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:55:49.451123 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-2bd2852d-cc5e-4a16-9b65-f126372b216e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:49.451783 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-2bd2852d-cc5e-4a16-9b65-f126372b216e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:49.453666 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1373/2762] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:55:49 2022] DELETE /identity/v3/projects/d32636d2103a45d8ac1902c8321b4411 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:55:49.457342 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2852d-cc5e-4a16-9b65-f126372b216e admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:55:49.458661 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-2bd2852d-cc5e-4a16-9b65-f126372b216e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:55:49.478621 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372579de-0500-4c1d-8442-fa1244ee9c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:55:49.479304 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372579de-0500-4c1d-8442-fa1244ee9c1d admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:49.479594 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372579de-0500-4c1d-8442-fa1244ee9c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:49.479878 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372579de-0500-4c1d-8442-fa1244ee9c1d admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:49.480617 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372579de-0500-4c1d-8442-fa1244ee9c1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:55:49.480934 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372579de-0500-4c1d-8442-fa1244ee9c1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:55:49.482620 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372579de-0500-4c1d-8442-fa1244ee9c1d admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:55:49.494954 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1390/2763] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:55:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 16:55:49.499103 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372579de-0500-4c1d-8442-fa1244ee9c1d admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:55:49.500755 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-372579de-0500-4c1d-8442-fa1244ee9c1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:55:49.513398 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372579de-0500-4c1d-8442-fa1244ee9c1d admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:55:49.518164 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620127f6-eb63-4a5a-b7de-60334b83e63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:55:49.518813 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620127f6-eb63-4a5a-b7de-60334b83e63a admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:49.519103 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620127f6-eb63-4a5a-b7de-60334b83e63a admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:49.519386 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620127f6-eb63-4a5a-b7de-60334b83e63a admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:49.520080 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620127f6-eb63-4a5a-b7de-60334b83e63a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:55:49.522028 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620127f6-eb63-4a5a-b7de-60334b83e63a admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:55:49.529087 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2852d-cc5e-4a16-9b65-f126372b216e admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:55:49.533808 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f77c79-18be-470b-8187-d1ddd3c01cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:55:49.534690 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f77c79-18be-470b-8187-d1ddd3c01cc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:49.534979 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f77c79-18be-470b-8187-d1ddd3c01cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:49.535260 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f77c79-18be-470b-8187-d1ddd3c01cc4 admin admin] PATH_INFO: `/v3/users/09befbde89c84818ae570376bc228a8b` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:49.536476 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f77c79-18be-470b-8187-d1ddd3c01cc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09befbde89c84818ae570376bc228a8b)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:55:49.538429 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f77c79-18be-470b-8187-d1ddd3c01cc4 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:55:49.548704 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1391/2764] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:55:49 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:55:49.552787 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620127f6-eb63-4a5a-b7de-60334b83e63a admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:55:49.561390 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:55:49.587789 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-54f77c79-18be-470b-8187-d1ddd3c01cc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09befbde89c84818ae570376bc228a8b'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:49.588498 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-54f77c79-18be-470b-8187-d1ddd3c01cc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09befbde89c84818ae570376bc228a8b'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:49.596104 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-54f77c79-18be-470b-8187-d1ddd3c01cc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09befbde89c84818ae570376bc228a8b'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:49.608756 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-54f77c79-18be-470b-8187-d1ddd3c01cc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09befbde89c84818ae570376bc228a8b'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:49.609590 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-54f77c79-18be-470b-8187-d1ddd3c01cc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09befbde89c84818ae570376bc228a8b'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:49.619666 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620127f6-eb63-4a5a-b7de-60334b83e63a admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:55:49.622797 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1374/2765] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:55:49 2022] DELETE /identity/v3/users/09befbde89c84818ae570376bc228a8b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:55:49.625173 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fee371-02a2-44da-bb20-b031a82585fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:55:49.625920 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fee371-02a2-44da-bb20-b031a82585fc admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:49.626207 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fee371-02a2-44da-bb20-b031a82585fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:49.626486 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fee371-02a2-44da-bb20-b031a82585fc admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:49.627211 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fee371-02a2-44da-bb20-b031a82585fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:55:49.629183 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fee371-02a2-44da-bb20-b031a82585fc admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:55:49.651800 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1392/2766] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:55:49 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:55:49.656004 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f77c79-18be-470b-8187-d1ddd3c01cc4 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:55:49.657657 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-54f77c79-18be-470b-8187-d1ddd3c01cc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:55:49.691496 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:55:49.734964 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f77c79-18be-470b-8187-d1ddd3c01cc4 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:55:49.741592 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f83ec93-adae-457a-a7ab-57ee5ae7c5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:55:49.742698 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83ec93-adae-457a-a7ab-57ee5ae7c5d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:49.743218 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83ec93-adae-457a-a7ab-57ee5ae7c5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:49.743993 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83ec93-adae-457a-a7ab-57ee5ae7c5d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:49.745078 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f83ec93-adae-457a-a7ab-57ee5ae7c5d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:55:49.747039 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f83ec93-adae-457a-a7ab-57ee5ae7c5d4 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:55:49.762129 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1375/2767] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:55:49.763417 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fee371-02a2-44da-bb20-b031a82585fc admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:55:49.775510 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fee371-02a2-44da-bb20-b031a82585fc admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:55:49.781071 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead0cc4-91d4-475b-ae0f-9fc077462f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:55:49.781871 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead0cc4-91d4-475b-ae0f-9fc077462f9e admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:49.782111 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead0cc4-91d4-475b-ae0f-9fc077462f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:49.782525 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead0cc4-91d4-475b-ae0f-9fc077462f9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:49.783556 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bead0cc4-91d4-475b-ae0f-9fc077462f9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:55:49.785331 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bead0cc4-91d4-475b-ae0f-9fc077462f9e admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:55:49.815894 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1393/2768] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:55:49.817307 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f83ec93-adae-457a-a7ab-57ee5ae7c5d4 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:55:49.827735 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f83ec93-adae-457a-a7ab-57ee5ae7c5d4 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:55:49.832466 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2634cc38-3d7c-496a-a2a3-24d43d269948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:55:49.833371 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2634cc38-3d7c-496a-a2a3-24d43d269948 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:49.833661 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2634cc38-3d7c-496a-a2a3-24d43d269948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:49.833946 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2634cc38-3d7c-496a-a2a3-24d43d269948 admin admin] PATH_INFO: `/v3/projects/1eec3a9139624be69542440b311ee39d/users/bfac38790d0b4a288e91ca2b22f03477/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:49.850915 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2634cc38-3d7c-496a-a2a3-24d43d269948 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1eec3a9139624be69542440b311ee39d, user_id=bfac38790d0b4a288e91ca2b22f03477, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:55:49.854263 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2634cc38-3d7c-496a-a2a3-24d43d269948 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:55:49.867205 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1376/2769] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:55:49 2022] PUT /identity/v3/projects/1eec3a9139624be69542440b311ee39d/users/bfac38790d0b4a288e91ca2b22f03477/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:55:49.930712 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea37da-539d-4855-9922-f1df3ab29ada None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:49.931175 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea37da-539d-4855-9922-f1df3ab29ada None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:49.931263 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea37da-539d-4855-9922-f1df3ab29ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:49.958581 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-2cea37da-539d-4855-9922-f1df3ab29ada None None] MFA Rules not processed for user `bfac38790d0b4a288e91ca2b22f03477`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:55:49.959740 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-2cea37da-539d-4855-9922-f1df3ab29ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:55:50.026067 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1394/2770] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:55:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:55:50.037298 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2634cc38-3d7c-496a-a2a3-24d43d269948 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:55:50.039001 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-2634cc38-3d7c-496a-a2a3-24d43d269948 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:55:50.082276 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cea37da-539d-4855-9922-f1df3ab29ada None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:55:50.083711 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-2cea37da-539d-4855-9922-f1df3ab29ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:55:50.115471 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2634cc38-3d7c-496a-a2a3-24d43d269948 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:55:50.120680 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03039e4-fb64-4c0d-8816-42883e9d8a3e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57a5d7151478434f86be085bcc487f0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:55:50.121667 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03039e4-fb64-4c0d-8816-42883e9d8a3e service glance] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:50.121953 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03039e4-fb64-4c0d-8816-42883e9d8a3e service glance] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:50.122227 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03039e4-fb64-4c0d-8816-42883e9d8a3e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:50.135148 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03039e4-fb64-4c0d-8816-42883e9d8a3e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:55:50.136965 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03039e4-fb64-4c0d-8816-42883e9d8a3e service glance] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:55:50.148208 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1377/2771] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:55:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:55:50.167381 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cea37da-539d-4855-9922-f1df3ab29ada None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:55:50.176430 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb22ccd-55ae-4aca-be23-f844c075eeb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:55:50.177423 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb22ccd-55ae-4aca-be23-f844c075eeb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:50.177804 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb22ccd-55ae-4aca-be23-f844c075eeb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:50.178172 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb22ccd-55ae-4aca-be23-f844c075eeb0 admin admin] PATH_INFO: `/v3/projects/fa08f503e4d345619cf6ecb35eca7ca4` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:50.179547 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb22ccd-55ae-4aca-be23-f844c075eeb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa08f503e4d345619cf6ecb35eca7ca4)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:55:50.182185 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb22ccd-55ae-4aca-be23-f844c075eeb0 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:55:50.202104 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:55:50.237655 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-dcb22ccd-55ae-4aca-be23-f844c075eeb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa08f503e4d345619cf6ecb35eca7ca4', 'request_id': 'req-dcb22ccd-55ae-4aca-be23-f844c075eeb0'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:50.244065 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-dcb22ccd-55ae-4aca-be23-f844c075eeb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa08f503e4d345619cf6ecb35eca7ca4', 'request_id': 'req-dcb22ccd-55ae-4aca-be23-f844c075eeb0'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:50.251526 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-dcb22ccd-55ae-4aca-be23-f844c075eeb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa08f503e4d345619cf6ecb35eca7ca4', 'request_id': 'req-dcb22ccd-55ae-4aca-be23-f844c075eeb0'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:50.266285 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-dcb22ccd-55ae-4aca-be23-f844c075eeb0 admin admin] The token cache is being invalidate because project fa08f503e4d345619cf6ecb35eca7ca4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:55:50.267196 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-dcb22ccd-55ae-4aca-be23-f844c075eeb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:50.267891 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-dcb22ccd-55ae-4aca-be23-f844c075eeb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:55:50.269687 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1395/2772] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:55:50 2022] DELETE /identity/v3/projects/fa08f503e4d345619cf6ecb35eca7ca4 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:55:50.275817 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae93a91-07b1-435d-9fbe-7bd85fd609a4 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:50.276180 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae93a91-07b1-435d-9fbe-7bd85fd609a4 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:50.276530 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae93a91-07b1-435d-9fbe-7bd85fd609a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:50.298920 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-7ae93a91-07b1-435d-9fbe-7bd85fd609a4 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:55:50.300777 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-7ae93a91-07b1-435d-9fbe-7bd85fd609a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:55:50.372558 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1378/2773] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:55:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:55:50.380875 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec9ecc-3466-42e2-939a-55aacd1344d0 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:50.381200 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec9ecc-3466-42e2-939a-55aacd1344d0 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:50.381480 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec9ecc-3466-42e2-939a-55aacd1344d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:50.402021 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-ddec9ecc-3466-42e2-939a-55aacd1344d0 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:55:50.403246 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-ddec9ecc-3466-42e2-939a-55aacd1344d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:55:50.411628 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1396/2774] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:55:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:55:50.416098 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae93a91-07b1-435d-9fbe-7bd85fd609a4 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:55:50.427095 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae93a91-07b1-435d-9fbe-7bd85fd609a4 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:55:50.432427 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdf13bc-d5d9-43b7-8fa3-1b120a45066a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:55:50.433104 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf13bc-d5d9-43b7-8fa3-1b120a45066a admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:50.442541 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf13bc-d5d9-43b7-8fa3-1b120a45066a admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:50.442892 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf13bc-d5d9-43b7-8fa3-1b120a45066a admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:50.443668 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdf13bc-d5d9-43b7-8fa3-1b120a45066a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:55:50.443986 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdf13bc-d5d9-43b7-8fa3-1b120a45066a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:55:50.446300 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdf13bc-d5d9-43b7-8fa3-1b120a45066a admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:55:50.460017 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1379/2775] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:55:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:55:50.465754 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec9ecc-3466-42e2-939a-55aacd1344d0 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:55:50.476118 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec9ecc-3466-42e2-939a-55aacd1344d0 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:55:50.481495 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bc229f-d4ba-40df-bca8-f975a1286d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:55:50.482160 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc229f-d4ba-40df-bca8-f975a1286d81 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:50.482519 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc229f-d4ba-40df-bca8-f975a1286d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:50.482809 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc229f-d4ba-40df-bca8-f975a1286d81 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:50.483506 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bc229f-d4ba-40df-bca8-f975a1286d81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:55:50.485472 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bc229f-d4ba-40df-bca8-f975a1286d81 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:55:50.515583 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1397/2776] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:55:50 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:55:50.516771 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdf13bc-d5d9-43b7-8fa3-1b120a45066a admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:55:50.527692 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdf13bc-d5d9-43b7-8fa3-1b120a45066a admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:55:50.555799 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:55:50.593753 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338babac-2aab-4dc6-a75f-4457f5a048c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:55:50.594445 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338babac-2aab-4dc6-a75f-4457f5a048c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:50.594734 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338babac-2aab-4dc6-a75f-4457f5a048c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:50.595018 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338babac-2aab-4dc6-a75f-4457f5a048c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:50.595913 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338babac-2aab-4dc6-a75f-4457f5a048c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:55:50.597673 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338babac-2aab-4dc6-a75f-4457f5a048c9 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:55:50.620744 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1380/2777] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:55:50 2022] POST /identity/v3/users => generated 290 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:55:50.626929 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bc229f-d4ba-40df-bca8-f975a1286d81 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:55:50.639727 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bc229f-d4ba-40df-bca8-f975a1286d81 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:55:50.646242 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83126cd7-4a9a-4202-9aa4-62a61c0aa75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:55:50.646970 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83126cd7-4a9a-4202-9aa4-62a61c0aa75e admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:50.647332 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83126cd7-4a9a-4202-9aa4-62a61c0aa75e admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:50.647813 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83126cd7-4a9a-4202-9aa4-62a61c0aa75e admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:50.648664 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83126cd7-4a9a-4202-9aa4-62a61c0aa75e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:55:50.650626 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83126cd7-4a9a-4202-9aa4-62a61c0aa75e admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:55:50.664878 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1398/2778] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:55:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:55:50.666172 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338babac-2aab-4dc6-a75f-4457f5a048c9 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:55:50.676776 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338babac-2aab-4dc6-a75f-4457f5a048c9 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:55:50.682422 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7625e3b7-c613-4cb0-91a7-e1f2605ee7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:55:50.683087 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625e3b7-c613-4cb0-91a7-e1f2605ee7b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:50.683406 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625e3b7-c613-4cb0-91a7-e1f2605ee7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:50.683691 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625e3b7-c613-4cb0-91a7-e1f2605ee7b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:50.684424 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7625e3b7-c613-4cb0-91a7-e1f2605ee7b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:55:50.686153 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7625e3b7-c613-4cb0-91a7-e1f2605ee7b7 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:55:50.696988 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1381/2779] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:55:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:55:50.701422 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83126cd7-4a9a-4202-9aa4-62a61c0aa75e admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:55:50.711384 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83126cd7-4a9a-4202-9aa4-62a61c0aa75e admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:55:50.716178 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d4d48-7e8c-476a-83da-46c6251b7b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:55:50.716991 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d4d48-7e8c-476a-83da-46c6251b7b3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:50.717319 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d4d48-7e8c-476a-83da-46c6251b7b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:50.717608 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d4d48-7e8c-476a-83da-46c6251b7b3b admin admin] PATH_INFO: `/v3/projects/ea859952b67847e595daa9ce81c3b76e/users/da7389b509484e8c809e54c321ffd211/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:50.732495 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d4d48-7e8c-476a-83da-46c6251b7b3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea859952b67847e595daa9ce81c3b76e, user_id=da7389b509484e8c809e54c321ffd211, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:55:50.735245 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d4d48-7e8c-476a-83da-46c6251b7b3b admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:55:50.748718 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1399/2780] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:55:50 2022] PUT /identity/v3/projects/ea859952b67847e595daa9ce81c3b76e/users/da7389b509484e8c809e54c321ffd211/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:55:50.761818 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3548c-855d-4f71-916d-630d5c1ebdef None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:50.762156 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3548c-855d-4f71-916d-630d5c1ebdef None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:50.762404 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3548c-855d-4f71-916d-630d5c1ebdef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:50.789231 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-fea3548c-855d-4f71-916d-630d5c1ebdef None None] MFA Rules not processed for user `da7389b509484e8c809e54c321ffd211`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:55:50.790478 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-fea3548c-855d-4f71-916d-630d5c1ebdef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:55:50.856831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1382/2781] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:55:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:55:50.861414 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d4d48-7e8c-476a-83da-46c6251b7b3b admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:55:50.932341 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d4d48-7e8c-476a-83da-46c6251b7b3b admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:55:50.937376 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f121c-5909-45f2-ab39-42890e04942b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:55:50.938210 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f121c-5909-45f2-ab39-42890e04942b admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:50.938498 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f121c-5909-45f2-ab39-42890e04942b admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:50.938777 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f121c-5909-45f2-ab39-42890e04942b admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:50.939502 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20f121c-5909-45f2-ab39-42890e04942b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:55:50.941538 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20f121c-5909-45f2-ab39-42890e04942b admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:55:50.967508 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1400/2782] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:55:50 2022] POST /identity/v3/projects => generated 379 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:55:50.971654 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea3548c-855d-4f71-916d-630d5c1ebdef None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:55:50.982897 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea3548c-855d-4f71-916d-630d5c1ebdef None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:55:51.040112 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fcf033-f8e0-4859-a285-4bae10fe0238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:55:51.040791 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fcf033-f8e0-4859-a285-4bae10fe0238 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:51.041085 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fcf033-f8e0-4859-a285-4bae10fe0238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:51.041396 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fcf033-f8e0-4859-a285-4bae10fe0238 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:51.042107 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fcf033-f8e0-4859-a285-4bae10fe0238 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:55:51.044030 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fcf033-f8e0-4859-a285-4bae10fe0238 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:55:51.065403 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1383/2783] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:55:50 2022] POST /identity/v3/users => generated 296 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:55:51.069482 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f121c-5909-45f2-ab39-42890e04942b admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:55:51.079817 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f121c-5909-45f2-ab39-42890e04942b admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:55:51.084493 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a529d3-b000-4366-ae12-8d6be23214cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:55:51.085178 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a529d3-b000-4366-ae12-8d6be23214cb admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:51.085475 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a529d3-b000-4366-ae12-8d6be23214cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:51.085760 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a529d3-b000-4366-ae12-8d6be23214cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:51.086538 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a529d3-b000-4366-ae12-8d6be23214cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:55:51.088187 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a529d3-b000-4366-ae12-8d6be23214cb admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:55:51.099253 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1401/2784] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:55:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:55:51.104028 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fcf033-f8e0-4859-a285-4bae10fe0238 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:55:51.114009 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fcf033-f8e0-4859-a285-4bae10fe0238 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:55:51.120289 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16aa073-d58e-4209-a0b4-f52501f2ef9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:55:51.121485 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16aa073-d58e-4209-a0b4-f52501f2ef9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:51.121768 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16aa073-d58e-4209-a0b4-f52501f2ef9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:51.122045 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16aa073-d58e-4209-a0b4-f52501f2ef9d admin admin] PATH_INFO: `/v3/projects/11bbbeb6fb5341828066ef72e822d09d/users/f83abdf4cb00474e95f80856336cc356/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:51.136892 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16aa073-d58e-4209-a0b4-f52501f2ef9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=11bbbeb6fb5341828066ef72e822d09d, user_id=f83abdf4cb00474e95f80856336cc356, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:55:51.140166 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16aa073-d58e-4209-a0b4-f52501f2ef9d admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:55:51.153354 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1384/2785] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:55:51 2022] PUT /identity/v3/projects/11bbbeb6fb5341828066ef72e822d09d/users/f83abdf4cb00474e95f80856336cc356/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:55:51.157255 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a529d3-b000-4366-ae12-8d6be23214cb admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:55:51.167785 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a529d3-b000-4366-ae12-8d6be23214cb admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:55:51.180495 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:55:51.227751 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15685ae-009a-4451-9545-9fb0448e488b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:55:51.228362 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15685ae-009a-4451-9545-9fb0448e488b admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:51.228636 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15685ae-009a-4451-9545-9fb0448e488b admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:51.228917 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15685ae-009a-4451-9545-9fb0448e488b admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:51.229699 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15685ae-009a-4451-9545-9fb0448e488b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:55:51.231349 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15685ae-009a-4451-9545-9fb0448e488b admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:55:51.241857 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1402/2786] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:55:51 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:55:51.245831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16aa073-d58e-4209-a0b4-f52501f2ef9d admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:55:51.251505 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:55:51.258265 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16aa073-d58e-4209-a0b4-f52501f2ef9d admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:55:51.264333 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa48c9a2-5d57-4022-b0a7-9c3e04d40f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:55:51.265265 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48c9a2-5d57-4022-b0a7-9c3e04d40f8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:51.265551 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48c9a2-5d57-4022-b0a7-9c3e04d40f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:51.265831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48c9a2-5d57-4022-b0a7-9c3e04d40f8a admin admin] PATH_INFO: `/v3/projects/11bbbeb6fb5341828066ef72e822d09d/users/f83abdf4cb00474e95f80856336cc356/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:51.267782 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa48c9a2-5d57-4022-b0a7-9c3e04d40f8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=11bbbeb6fb5341828066ef72e822d09d, user_id=f83abdf4cb00474e95f80856336cc356, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:55:51.270525 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa48c9a2-5d57-4022-b0a7-9c3e04d40f8a admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:55:51.281537 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1385/2787] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:55:51 2022] PUT /identity/v3/projects/11bbbeb6fb5341828066ef72e822d09d/users/f83abdf4cb00474e95f80856336cc356/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:55:51.292717 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94405624-1f64-4eb1-9ed8-20c43067c29d None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:51.293015 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94405624-1f64-4eb1-9ed8-20c43067c29d None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:51.293346 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94405624-1f64-4eb1-9ed8-20c43067c29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:51.319294 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-94405624-1f64-4eb1-9ed8-20c43067c29d None None] MFA Rules not processed for user `f83abdf4cb00474e95f80856336cc356`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:55:51.320555 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-94405624-1f64-4eb1-9ed8-20c43067c29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:55:51.390198 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1403/2788] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:55:51 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:55:51.402327 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa48c9a2-5d57-4022-b0a7-9c3e04d40f8a admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:55:51.404166 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-aa48c9a2-5d57-4022-b0a7-9c3e04d40f8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:55:51.472966 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa48c9a2-5d57-4022-b0a7-9c3e04d40f8a admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:55:51.477423 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc448ea6-a55d-49aa-92f2-931d5d608791 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:55:51.478174 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc448ea6-a55d-49aa-92f2-931d5d608791 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:51.478431 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc448ea6-a55d-49aa-92f2-931d5d608791 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:51.478714 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc448ea6-a55d-49aa-92f2-931d5d608791 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:51.488402 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc448ea6-a55d-49aa-92f2-931d5d608791 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:55:51.490030 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc448ea6-a55d-49aa-92f2-931d5d608791 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:55:51.501126 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1386/2789] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:55:51 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:55:51.578487 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94405624-1f64-4eb1-9ed8-20c43067c29d None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:55:51.589970 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94405624-1f64-4eb1-9ed8-20c43067c29d None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:55:51.594266 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2873e34-7912-46c2-b46d-cf2a853b4051 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:55:51.594924 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2873e34-7912-46c2-b46d-cf2a853b4051 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:55:51.595211 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2873e34-7912-46c2-b46d-cf2a853b4051 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:55:51.595490 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2873e34-7912-46c2-b46d-cf2a853b4051 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:55:51.674039 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2873e34-7912-46c2-b46d-cf2a853b4051 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:55:51.675513 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2873e34-7912-46c2-b46d-cf2a853b4051 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:55:51.685603 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1404/2790] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:55:51 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:56:01.498065 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:56:01.680673 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=84 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:56:04.588091 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc448ea6-a55d-49aa-92f2-931d5d608791 service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:04.648980 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc448ea6-a55d-49aa-92f2-931d5d608791 service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:04.654089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45bec2a-df3e-4734-8f03-59cd5de9deb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:04.654953 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45bec2a-df3e-4734-8f03-59cd5de9deb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:04.655239 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45bec2a-df3e-4734-8f03-59cd5de9deb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:04.655517 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45bec2a-df3e-4734-8f03-59cd5de9deb3 admin admin] PATH_INFO: `/v3/users/da7389b509484e8c809e54c321ffd211` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:04.656691 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45bec2a-df3e-4734-8f03-59cd5de9deb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da7389b509484e8c809e54c321ffd211)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:04.658715 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45bec2a-df3e-4734-8f03-59cd5de9deb3 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:04.702482 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-a45bec2a-df3e-4734-8f03-59cd5de9deb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da7389b509484e8c809e54c321ffd211'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:04.703145 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-a45bec2a-df3e-4734-8f03-59cd5de9deb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da7389b509484e8c809e54c321ffd211'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:04.709897 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-a45bec2a-df3e-4734-8f03-59cd5de9deb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da7389b509484e8c809e54c321ffd211'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:04.722015 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-a45bec2a-df3e-4734-8f03-59cd5de9deb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da7389b509484e8c809e54c321ffd211'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:04.722645 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-a45bec2a-df3e-4734-8f03-59cd5de9deb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da7389b509484e8c809e54c321ffd211'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:04.735029 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1387/2791] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:56:04 2022] DELETE /identity/v3/users/da7389b509484e8c809e54c321ffd211 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:56:04.739494 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643e109-08d3-4781-9d1a-7bdeb3226450 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:04.739973 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643e109-08d3-4781-9d1a-7bdeb3226450 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:04.740075 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643e109-08d3-4781-9d1a-7bdeb3226450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:04.756931 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-5643e109-08d3-4781-9d1a-7bdeb3226450 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:56:04.758099 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-5643e109-08d3-4781-9d1a-7bdeb3226450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:56:04.831416 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1405/2792] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:56:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:56:04.841557 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45bec2a-df3e-4734-8f03-59cd5de9deb3 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:04.843019 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-a45bec2a-df3e-4734-8f03-59cd5de9deb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:56:04.904644 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45bec2a-df3e-4734-8f03-59cd5de9deb3 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:04.908344 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43be2ab2-999c-4bfe-9655-538cf32fde51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:04.908997 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43be2ab2-999c-4bfe-9655-538cf32fde51 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:04.909330 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43be2ab2-999c-4bfe-9655-538cf32fde51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:04.909617 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43be2ab2-999c-4bfe-9655-538cf32fde51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:04.919660 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43be2ab2-999c-4bfe-9655-538cf32fde51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:04.921740 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43be2ab2-999c-4bfe-9655-538cf32fde51 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:04.932334 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1388/2793] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:56:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:56:05.205800 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5643e109-08d3-4781-9d1a-7bdeb3226450 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:05.207406 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-5643e109-08d3-4781-9d1a-7bdeb3226450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:56:05.222843 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5643e109-08d3-4781-9d1a-7bdeb3226450 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:05.228579 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac0d887-bd4b-48e9-b563-4821b9b12b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:05.229489 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac0d887-bd4b-48e9-b563-4821b9b12b21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:05.229839 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac0d887-bd4b-48e9-b563-4821b9b12b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:05.230124 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac0d887-bd4b-48e9-b563-4821b9b12b21 admin admin] PATH_INFO: `/v3/projects/ea859952b67847e595daa9ce81c3b76e` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:05.231287 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac0d887-bd4b-48e9-b563-4821b9b12b21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea859952b67847e595daa9ce81c3b76e)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:05.233244 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac0d887-bd4b-48e9-b563-4821b9b12b21 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:05.281045 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-7ac0d887-bd4b-48e9-b563-4821b9b12b21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea859952b67847e595daa9ce81c3b76e', 'request_id': 'req-7ac0d887-bd4b-48e9-b563-4821b9b12b21'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:05.286639 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-7ac0d887-bd4b-48e9-b563-4821b9b12b21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea859952b67847e595daa9ce81c3b76e', 'request_id': 'req-7ac0d887-bd4b-48e9-b563-4821b9b12b21'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:05.292915 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-7ac0d887-bd4b-48e9-b563-4821b9b12b21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea859952b67847e595daa9ce81c3b76e', 'request_id': 'req-7ac0d887-bd4b-48e9-b563-4821b9b12b21'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:05.304933 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-7ac0d887-bd4b-48e9-b563-4821b9b12b21 admin admin] The token cache is being invalidate because project ea859952b67847e595daa9ce81c3b76e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:56:05.305242 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-7ac0d887-bd4b-48e9-b563-4821b9b12b21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:05.306002 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-7ac0d887-bd4b-48e9-b563-4821b9b12b21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:05.308229 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1406/2794] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:56:05 2022] DELETE /identity/v3/projects/ea859952b67847e595daa9ce81c3b76e => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:56:05.312169 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43be2ab2-999c-4bfe-9655-538cf32fde51 service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:05.324302 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43be2ab2-999c-4bfe-9655-538cf32fde51 service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:05.345563 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:56:05.389072 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0c40fa-5871-4b9c-b543-906bca095fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:05.390120 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c40fa-5871-4b9c-b543-906bca095fef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:05.390408 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c40fa-5871-4b9c-b543-906bca095fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:05.390689 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c40fa-5871-4b9c-b543-906bca095fef admin admin] PATH_INFO: `/v3/users/f83abdf4cb00474e95f80856336cc356` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:05.391911 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0c40fa-5871-4b9c-b543-906bca095fef admin admin] RBAC: Authorizing `identity:delete_user(user_id=f83abdf4cb00474e95f80856336cc356)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:05.394178 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0c40fa-5871-4b9c-b543-906bca095fef admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:05.456944 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-9d0c40fa-5871-4b9c-b543-906bca095fef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f83abdf4cb00474e95f80856336cc356'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:05.457190 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-9d0c40fa-5871-4b9c-b543-906bca095fef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f83abdf4cb00474e95f80856336cc356'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:05.465982 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-9d0c40fa-5871-4b9c-b543-906bca095fef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f83abdf4cb00474e95f80856336cc356'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:05.480433 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-9d0c40fa-5871-4b9c-b543-906bca095fef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f83abdf4cb00474e95f80856336cc356'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:05.481114 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-9d0c40fa-5871-4b9c-b543-906bca095fef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f83abdf4cb00474e95f80856336cc356'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:05.493691 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1389/2795] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:56:05 2022] DELETE /identity/v3/users/f83abdf4cb00474e95f80856336cc356 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:56:05.832109 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac0d887-bd4b-48e9-b563-4821b9b12b21 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:05.833777 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-7ac0d887-bd4b-48e9-b563-4821b9b12b21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:56:05.847873 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac0d887-bd4b-48e9-b563-4821b9b12b21 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:05.911270 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e717a-2aa2-485b-bac7-0de8672206a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:05.912132 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e717a-2aa2-485b-bac7-0de8672206a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:05.912415 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e717a-2aa2-485b-bac7-0de8672206a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:05.912690 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e717a-2aa2-485b-bac7-0de8672206a1 admin admin] PATH_INFO: `/v3/projects/11bbbeb6fb5341828066ef72e822d09d` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:05.913877 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0e717a-2aa2-485b-bac7-0de8672206a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11bbbeb6fb5341828066ef72e822d09d)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:05.915779 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0e717a-2aa2-485b-bac7-0de8672206a1 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:05.945271 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:56:05.964797 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6a0e717a-2aa2-485b-bac7-0de8672206a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11bbbeb6fb5341828066ef72e822d09d', 'request_id': 'req-6a0e717a-2aa2-485b-bac7-0de8672206a1'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:05.970710 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6a0e717a-2aa2-485b-bac7-0de8672206a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11bbbeb6fb5341828066ef72e822d09d', 'request_id': 'req-6a0e717a-2aa2-485b-bac7-0de8672206a1'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:05.976809 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6a0e717a-2aa2-485b-bac7-0de8672206a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11bbbeb6fb5341828066ef72e822d09d', 'request_id': 'req-6a0e717a-2aa2-485b-bac7-0de8672206a1'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:05.988898 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6a0e717a-2aa2-485b-bac7-0de8672206a1 admin admin] The token cache is being invalidate because project 11bbbeb6fb5341828066ef72e822d09d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:56:05.989214 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6a0e717a-2aa2-485b-bac7-0de8672206a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:05.989872 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-6a0e717a-2aa2-485b-bac7-0de8672206a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:05.991417 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1407/2796] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:56:05 2022] DELETE /identity/v3/projects/11bbbeb6fb5341828066ef72e822d09d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:56:05.996529 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6547e11-2036-45f1-8b5b-60dfb8b977cc None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:05.997007 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6547e11-2036-45f1-8b5b-60dfb8b977cc None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:05.997106 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6547e11-2036-45f1-8b5b-60dfb8b977cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:06.015106 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-d6547e11-2036-45f1-8b5b-60dfb8b977cc None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:56:06.016326 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-d6547e11-2036-45f1-8b5b-60dfb8b977cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:56:06.074213 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1390/2797] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:56:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:56:06.082238 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d7d76-85b9-43dc-852a-7d9b33c12138 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:06.082476 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d7d76-85b9-43dc-852a-7d9b33c12138 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:06.082769 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d7d76-85b9-43dc-852a-7d9b33c12138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:06.101109 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-925d7d76-85b9-43dc-852a-7d9b33c12138 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:56:06.102330 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-925d7d76-85b9-43dc-852a-7d9b33c12138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:56:06.110058 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1408/2798] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:56:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:56:06.114131 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6547e11-2036-45f1-8b5b-60dfb8b977cc None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:06.123920 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6547e11-2036-45f1-8b5b-60dfb8b977cc None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:06.128553 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e566c-dcd9-422d-8ffb-5b4eaa7493bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:06.129239 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e566c-dcd9-422d-8ffb-5b4eaa7493bb admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:06.129533 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e566c-dcd9-422d-8ffb-5b4eaa7493bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:06.129818 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e566c-dcd9-422d-8ffb-5b4eaa7493bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:06.130552 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12e566c-dcd9-422d-8ffb-5b4eaa7493bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:56:06.130871 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12e566c-dcd9-422d-8ffb-5b4eaa7493bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:06.132469 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12e566c-dcd9-422d-8ffb-5b4eaa7493bb admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:06.144763 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1391/2799] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:56:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:56:06.148757 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d7d76-85b9-43dc-852a-7d9b33c12138 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:06.158351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d7d76-85b9-43dc-852a-7d9b33c12138 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:06.162914 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb0b2c-0ce9-4b61-9195-b198557af1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:06.163562 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb0b2c-0ce9-4b61-9195-b198557af1e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:06.163846 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb0b2c-0ce9-4b61-9195-b198557af1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:06.164121 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb0b2c-0ce9-4b61-9195-b198557af1e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:06.164829 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fb0b2c-0ce9-4b61-9195-b198557af1e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:06.166776 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fb0b2c-0ce9-4b61-9195-b198557af1e0 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:06.192358 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1409/2800] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:56:06 2022] POST /identity/v3/projects => generated 371 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:56:06.196627 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e566c-dcd9-422d-8ffb-5b4eaa7493bb admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:06.207318 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e566c-dcd9-422d-8ffb-5b4eaa7493bb admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:06.232409 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:56:06.265788 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9730137a-0744-48a5-b7ca-e5f6900f3204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:06.266469 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730137a-0744-48a5-b7ca-e5f6900f3204 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:06.266761 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730137a-0744-48a5-b7ca-e5f6900f3204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:06.267049 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730137a-0744-48a5-b7ca-e5f6900f3204 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:06.267750 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9730137a-0744-48a5-b7ca-e5f6900f3204 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:06.269713 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9730137a-0744-48a5-b7ca-e5f6900f3204 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:06.292309 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1392/2801] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:56:06 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:56:06.293502 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb0b2c-0ce9-4b61-9195-b198557af1e0 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:06.303705 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb0b2c-0ce9-4b61-9195-b198557af1e0 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:06.308972 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87de427-d2a3-4318-b3a5-15427fdcd55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:06.309668 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87de427-d2a3-4318-b3a5-15427fdcd55a admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:06.310222 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87de427-d2a3-4318-b3a5-15427fdcd55a admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:06.310510 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87de427-d2a3-4318-b3a5-15427fdcd55a admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:06.311243 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87de427-d2a3-4318-b3a5-15427fdcd55a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:06.312848 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87de427-d2a3-4318-b3a5-15427fdcd55a admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:06.323203 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1410/2802] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:56:06.327583 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9730137a-0744-48a5-b7ca-e5f6900f3204 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:06.337559 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9730137a-0744-48a5-b7ca-e5f6900f3204 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:06.342131 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294329f7-6bfd-4cef-8ebc-6c5e92ce0965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:06.342785 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294329f7-6bfd-4cef-8ebc-6c5e92ce0965 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:06.343077 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294329f7-6bfd-4cef-8ebc-6c5e92ce0965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:06.343363 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294329f7-6bfd-4cef-8ebc-6c5e92ce0965 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:06.344091 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294329f7-6bfd-4cef-8ebc-6c5e92ce0965 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:06.345733 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294329f7-6bfd-4cef-8ebc-6c5e92ce0965 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:06.356431 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1393/2803] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:56:06.359229 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87de427-d2a3-4318-b3a5-15427fdcd55a admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:06.369213 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87de427-d2a3-4318-b3a5-15427fdcd55a admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:06.374830 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35518659-2aab-423e-bfe5-3eade0f7c62f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:06.375842 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35518659-2aab-423e-bfe5-3eade0f7c62f admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:06.376130 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35518659-2aab-423e-bfe5-3eade0f7c62f admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:06.376416 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35518659-2aab-423e-bfe5-3eade0f7c62f admin admin] PATH_INFO: `/v3/projects/28def19a29f0470c8dbe499152ca357d/users/c0e81e5a808d445d888b735c77bea107/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:06.394871 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35518659-2aab-423e-bfe5-3eade0f7c62f admin admin] RBAC: Authorizing `identity:create_grant(project_id=28def19a29f0470c8dbe499152ca357d, user_id=c0e81e5a808d445d888b735c77bea107, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:06.397634 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35518659-2aab-423e-bfe5-3eade0f7c62f admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:06.422482 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1411/2804] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:56:06 2022] PUT /identity/v3/projects/28def19a29f0470c8dbe499152ca357d/users/c0e81e5a808d445d888b735c77bea107/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:56:06.424585 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870386a-2608-48ab-ae12-bb445027a252 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:06.424900 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870386a-2608-48ab-ae12-bb445027a252 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:06.425214 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870386a-2608-48ab-ae12-bb445027a252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:06.455555 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-5870386a-2608-48ab-ae12-bb445027a252 None None] MFA Rules not processed for user `c0e81e5a808d445d888b735c77bea107`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:56:06.456842 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-5870386a-2608-48ab-ae12-bb445027a252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:56:06.529720 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1394/2805] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:56:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:56:06.534248 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35518659-2aab-423e-bfe5-3eade0f7c62f admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:06.596447 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35518659-2aab-423e-bfe5-3eade0f7c62f admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:06.601209 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a393a700-c155-40fe-81ba-cd9c971af87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:06.601870 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393a700-c155-40fe-81ba-cd9c971af87b admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:06.602161 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393a700-c155-40fe-81ba-cd9c971af87b admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:06.602442 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393a700-c155-40fe-81ba-cd9c971af87b admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:06.603131 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a393a700-c155-40fe-81ba-cd9c971af87b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:06.605624 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a393a700-c155-40fe-81ba-cd9c971af87b admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:06.631557 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1412/2806] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:56:06 2022] POST /identity/v3/projects => generated 371 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:56:06.635668 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5870386a-2608-48ab-ae12-bb445027a252 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:06.646185 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5870386a-2608-48ab-ae12-bb445027a252 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:06.702595 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6716f7b8-dd0c-496a-ae2e-22ae3c7e3426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:06.703266 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716f7b8-dd0c-496a-ae2e-22ae3c7e3426 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:06.703557 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716f7b8-dd0c-496a-ae2e-22ae3c7e3426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:06.703841 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716f7b8-dd0c-496a-ae2e-22ae3c7e3426 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:06.704532 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6716f7b8-dd0c-496a-ae2e-22ae3c7e3426 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:06.706507 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6716f7b8-dd0c-496a-ae2e-22ae3c7e3426 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:06.726174 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1395/2807] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:56:06 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:56:06.730611 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a393a700-c155-40fe-81ba-cd9c971af87b admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:06.740581 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a393a700-c155-40fe-81ba-cd9c971af87b admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:06.746248 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34993f35-1b05-4a7e-801b-f4177b5ba651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:06.746898 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34993f35-1b05-4a7e-801b-f4177b5ba651 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:06.747767 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34993f35-1b05-4a7e-801b-f4177b5ba651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:06.747767 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34993f35-1b05-4a7e-801b-f4177b5ba651 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:06.748157 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34993f35-1b05-4a7e-801b-f4177b5ba651 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:06.749785 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34993f35-1b05-4a7e-801b-f4177b5ba651 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:06.763512 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1413/2808] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:56:06.764646 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6716f7b8-dd0c-496a-ae2e-22ae3c7e3426 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:06.775464 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6716f7b8-dd0c-496a-ae2e-22ae3c7e3426 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:06.785909 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141781d3-b297-4ba5-b70f-e021c0e7e719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:06.786815 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141781d3-b297-4ba5-b70f-e021c0e7e719 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:06.787104 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141781d3-b297-4ba5-b70f-e021c0e7e719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:06.787549 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141781d3-b297-4ba5-b70f-e021c0e7e719 admin admin] PATH_INFO: `/v3/projects/a7bfec8e64f9495dab42ce2546c30bce/users/c4f4a27fda204b89b594cf3cc544cd8f/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:06.810089 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141781d3-b297-4ba5-b70f-e021c0e7e719 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7bfec8e64f9495dab42ce2546c30bce, user_id=c4f4a27fda204b89b594cf3cc544cd8f, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:06.812764 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141781d3-b297-4ba5-b70f-e021c0e7e719 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:06.827715 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1396/2809] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:56:06 2022] PUT /identity/v3/projects/a7bfec8e64f9495dab42ce2546c30bce/users/c4f4a27fda204b89b594cf3cc544cd8f/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:56:06.832206 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34993f35-1b05-4a7e-801b-f4177b5ba651 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:06.842803 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34993f35-1b05-4a7e-801b-f4177b5ba651 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:06.863882 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:56:06.926674 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4010924e-7fee-42b5-b036-d3504b5054e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:06.926674 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010924e-7fee-42b5-b036-d3504b5054e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:06.926674 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010924e-7fee-42b5-b036-d3504b5054e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:06.926674 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010924e-7fee-42b5-b036-d3504b5054e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:06.927098 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4010924e-7fee-42b5-b036-d3504b5054e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:06.927200 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4010924e-7fee-42b5-b036-d3504b5054e2 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:06.939036 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1414/2810] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:56:06.943422 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141781d3-b297-4ba5-b70f-e021c0e7e719 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:06.951429 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:56:06.957765 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141781d3-b297-4ba5-b70f-e021c0e7e719 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:06.967595 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb7ea28-98c1-40ff-b075-bb5b300446db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:06.968634 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7ea28-98c1-40ff-b075-bb5b300446db admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:06.969040 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7ea28-98c1-40ff-b075-bb5b300446db admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:06.969352 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7ea28-98c1-40ff-b075-bb5b300446db admin admin] PATH_INFO: `/v3/projects/a7bfec8e64f9495dab42ce2546c30bce/users/c4f4a27fda204b89b594cf3cc544cd8f/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:06.971689 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb7ea28-98c1-40ff-b075-bb5b300446db admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7bfec8e64f9495dab42ce2546c30bce, user_id=c4f4a27fda204b89b594cf3cc544cd8f, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:06.974576 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb7ea28-98c1-40ff-b075-bb5b300446db admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:06.987029 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1397/2811] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:56:06 2022] PUT /identity/v3/projects/a7bfec8e64f9495dab42ce2546c30bce/users/c4f4a27fda204b89b594cf3cc544cd8f/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:56:06.999769 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4e363-06d4-4f95-b947-898b9f5bfce7 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:07.000320 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4e363-06d4-4f95-b947-898b9f5bfce7 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:07.000871 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4e363-06d4-4f95-b947-898b9f5bfce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:07.031599 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-c4c4e363-06d4-4f95-b947-898b9f5bfce7 None None] MFA Rules not processed for user `c4f4a27fda204b89b594cf3cc544cd8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:56:07.032810 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-c4c4e363-06d4-4f95-b947-898b9f5bfce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:56:07.105055 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1415/2812] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:56:06 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:56:07.117393 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb7ea28-98c1-40ff-b075-bb5b300446db admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:07.119411 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-4fb7ea28-98c1-40ff-b075-bb5b300446db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:56:07.186046 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb7ea28-98c1-40ff-b075-bb5b300446db admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:07.189766 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d7c07f-5e83-4ea9-b157-0865b3bd709b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:07.190430 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d7c07f-5e83-4ea9-b157-0865b3bd709b service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:07.190720 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d7c07f-5e83-4ea9-b157-0865b3bd709b service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:07.191006 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d7c07f-5e83-4ea9-b157-0865b3bd709b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:07.201398 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d7c07f-5e83-4ea9-b157-0865b3bd709b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:07.202766 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d7c07f-5e83-4ea9-b157-0865b3bd709b service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:07.212392 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1398/2813] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:56:07 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:56:07.572506 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c4e363-06d4-4f95-b947-898b9f5bfce7 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:07.636728 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c4e363-06d4-4f95-b947-898b9f5bfce7 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:07.641294 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3b1b5d-283a-47f8-b211-69dff2fb2470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:07.642165 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b1b5d-283a-47f8-b211-69dff2fb2470 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:07.642451 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b1b5d-283a-47f8-b211-69dff2fb2470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:07.642734 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b1b5d-283a-47f8-b211-69dff2fb2470 admin admin] PATH_INFO: `/v3/users/c0e81e5a808d445d888b735c77bea107` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:07.643929 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3b1b5d-283a-47f8-b211-69dff2fb2470 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0e81e5a808d445d888b735c77bea107)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:07.645899 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3b1b5d-283a-47f8-b211-69dff2fb2470 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:07.690818 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ac3b1b5d-283a-47f8-b211-69dff2fb2470 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0e81e5a808d445d888b735c77bea107'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:07.691476 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ac3b1b5d-283a-47f8-b211-69dff2fb2470 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0e81e5a808d445d888b735c77bea107'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:07.692078 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ac3b1b5d-283a-47f8-b211-69dff2fb2470 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0e81e5a808d445d888b735c77bea107'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:07.693748 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:56:07.701301 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ac3b1b5d-283a-47f8-b211-69dff2fb2470 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0e81e5a808d445d888b735c77bea107'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:07.714896 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-ac3b1b5d-283a-47f8-b211-69dff2fb2470 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0e81e5a808d445d888b735c77bea107'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:07.727349 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1416/2814] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:56:07 2022] DELETE /identity/v3/users/c0e81e5a808d445d888b735c77bea107 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:56:07.731746 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf245e6-2ec2-4ad2-9e4d-3e930e46fcc2 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:07.732040 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf245e6-2ec2-4ad2-9e4d-3e930e46fcc2 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:07.732321 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf245e6-2ec2-4ad2-9e4d-3e930e46fcc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:07.750659 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-faf245e6-2ec2-4ad2-9e4d-3e930e46fcc2 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:56:07.751865 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-faf245e6-2ec2-4ad2-9e4d-3e930e46fcc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:56:07.809982 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1399/2815] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:56:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:56:07.819576 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3b1b5d-283a-47f8-b211-69dff2fb2470 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:07.820952 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-ac3b1b5d-283a-47f8-b211-69dff2fb2470 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:56:07.878077 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3b1b5d-283a-47f8-b211-69dff2fb2470 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:07.881768 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a78fc67-00ed-4208-a92d-3196aea5b6ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:07.882416 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78fc67-00ed-4208-a92d-3196aea5b6ff service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:07.882706 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78fc67-00ed-4208-a92d-3196aea5b6ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:07.882987 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78fc67-00ed-4208-a92d-3196aea5b6ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:07.893989 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a78fc67-00ed-4208-a92d-3196aea5b6ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:07.895211 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a78fc67-00ed-4208-a92d-3196aea5b6ff service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:07.904908 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1417/2816] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:56:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:56:08.170681 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf245e6-2ec2-4ad2-9e4d-3e930e46fcc2 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:08.174167 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-faf245e6-2ec2-4ad2-9e4d-3e930e46fcc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:56:08.191157 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf245e6-2ec2-4ad2-9e4d-3e930e46fcc2 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:08.202917 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77715ae2-c240-40a8-825a-d85db0b1d4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:08.203807 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77715ae2-c240-40a8-825a-d85db0b1d4d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:08.204094 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77715ae2-c240-40a8-825a-d85db0b1d4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:08.204392 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77715ae2-c240-40a8-825a-d85db0b1d4d2 admin admin] PATH_INFO: `/v3/projects/28def19a29f0470c8dbe499152ca357d` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:08.205645 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77715ae2-c240-40a8-825a-d85db0b1d4d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28def19a29f0470c8dbe499152ca357d)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:08.207565 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77715ae2-c240-40a8-825a-d85db0b1d4d2 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:08.254568 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-77715ae2-c240-40a8-825a-d85db0b1d4d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28def19a29f0470c8dbe499152ca357d', 'request_id': 'req-77715ae2-c240-40a8-825a-d85db0b1d4d2'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:08.259982 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-77715ae2-c240-40a8-825a-d85db0b1d4d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28def19a29f0470c8dbe499152ca357d', 'request_id': 'req-77715ae2-c240-40a8-825a-d85db0b1d4d2'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:08.266067 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-77715ae2-c240-40a8-825a-d85db0b1d4d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28def19a29f0470c8dbe499152ca357d', 'request_id': 'req-77715ae2-c240-40a8-825a-d85db0b1d4d2'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:08.277869 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-77715ae2-c240-40a8-825a-d85db0b1d4d2 admin admin] The token cache is being invalidate because project 28def19a29f0470c8dbe499152ca357d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:56:08.278163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-77715ae2-c240-40a8-825a-d85db0b1d4d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:08.278811 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-77715ae2-c240-40a8-825a-d85db0b1d4d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:08.280355 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1400/2817] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:56:08 2022] DELETE /identity/v3/projects/28def19a29f0470c8dbe499152ca357d => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:56:08.284413 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a78fc67-00ed-4208-a92d-3196aea5b6ff service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:08.294752 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a78fc67-00ed-4208-a92d-3196aea5b6ff service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:08.350157 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc085c06-4beb-4dc2-9421-c217c4a357fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:08.351019 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc085c06-4beb-4dc2-9421-c217c4a357fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:08.351337 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc085c06-4beb-4dc2-9421-c217c4a357fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:08.351616 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc085c06-4beb-4dc2-9421-c217c4a357fd admin admin] PATH_INFO: `/v3/users/c4f4a27fda204b89b594cf3cc544cd8f` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:08.352792 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc085c06-4beb-4dc2-9421-c217c4a357fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4f4a27fda204b89b594cf3cc544cd8f)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:08.355590 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc085c06-4beb-4dc2-9421-c217c4a357fd admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:08.400890 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-dc085c06-4beb-4dc2-9421-c217c4a357fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4f4a27fda204b89b594cf3cc544cd8f'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:08.402136 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-dc085c06-4beb-4dc2-9421-c217c4a357fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4f4a27fda204b89b594cf3cc544cd8f'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:08.402737 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-dc085c06-4beb-4dc2-9421-c217c4a357fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4f4a27fda204b89b594cf3cc544cd8f'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:08.409847 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-dc085c06-4beb-4dc2-9421-c217c4a357fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4f4a27fda204b89b594cf3cc544cd8f'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:08.421365 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:56:08.422587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-dc085c06-4beb-4dc2-9421-c217c4a357fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4f4a27fda204b89b594cf3cc544cd8f'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:08.436638 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1418/2818] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:56:08 2022] DELETE /identity/v3/users/c4f4a27fda204b89b594cf3cc544cd8f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:56:08.699902 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77715ae2-c240-40a8-825a-d85db0b1d4d2 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:08.701389 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-77715ae2-c240-40a8-825a-d85db0b1d4d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:56:08.719394 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77715ae2-c240-40a8-825a-d85db0b1d4d2 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:08.732306 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:56:08.780055 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11efd53c-b7bf-4716-821a-86b31d433d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:08.780922 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efd53c-b7bf-4716-821a-86b31d433d4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:08.781234 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efd53c-b7bf-4716-821a-86b31d433d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:08.781514 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efd53c-b7bf-4716-821a-86b31d433d4d admin admin] PATH_INFO: `/v3/projects/a7bfec8e64f9495dab42ce2546c30bce` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:08.782690 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11efd53c-b7bf-4716-821a-86b31d433d4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7bfec8e64f9495dab42ce2546c30bce)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:08.784778 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11efd53c-b7bf-4716-821a-86b31d433d4d admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:08.831250 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-11efd53c-b7bf-4716-821a-86b31d433d4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7bfec8e64f9495dab42ce2546c30bce', 'request_id': 'req-11efd53c-b7bf-4716-821a-86b31d433d4d'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:08.836726 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-11efd53c-b7bf-4716-821a-86b31d433d4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7bfec8e64f9495dab42ce2546c30bce', 'request_id': 'req-11efd53c-b7bf-4716-821a-86b31d433d4d'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:08.843514 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-11efd53c-b7bf-4716-821a-86b31d433d4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7bfec8e64f9495dab42ce2546c30bce', 'request_id': 'req-11efd53c-b7bf-4716-821a-86b31d433d4d'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:08.856100 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-11efd53c-b7bf-4716-821a-86b31d433d4d admin admin] The token cache is being invalidate because project a7bfec8e64f9495dab42ce2546c30bce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:56:08.856387 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-11efd53c-b7bf-4716-821a-86b31d433d4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:08.857003 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-11efd53c-b7bf-4716-821a-86b31d433d4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:08.858563 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1401/2819] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:56:08 2022] DELETE /identity/v3/projects/a7bfec8e64f9495dab42ce2546c30bce => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:56:08.863599 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955a2b2-0b28-47d1-a36a-05db9335ab0f None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:08.863902 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955a2b2-0b28-47d1-a36a-05db9335ab0f None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:08.864193 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955a2b2-0b28-47d1-a36a-05db9335ab0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:08.883256 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-c955a2b2-0b28-47d1-a36a-05db9335ab0f None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:56:08.884453 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-c955a2b2-0b28-47d1-a36a-05db9335ab0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:56:08.949689 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1419/2820] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:56:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:56:08.957800 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240e312-0b56-4a7c-af57-d6424f2d26e7 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:08.958096 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240e312-0b56-4a7c-af57-d6424f2d26e7 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:08.958378 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240e312-0b56-4a7c-af57-d6424f2d26e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:08.977634 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-6240e312-0b56-4a7c-af57-d6424f2d26e7 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:56:08.978854 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-6240e312-0b56-4a7c-af57-d6424f2d26e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:56:08.986790 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1402/2821] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:56:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:56:08.991029 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c955a2b2-0b28-47d1-a36a-05db9335ab0f None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:09.001253 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c955a2b2-0b28-47d1-a36a-05db9335ab0f None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:09.005908 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b09abf9-cae5-411a-96dc-9d6e65be3ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:09.006572 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b09abf9-cae5-411a-96dc-9d6e65be3ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:09.006859 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b09abf9-cae5-411a-96dc-9d6e65be3ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:09.007149 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b09abf9-cae5-411a-96dc-9d6e65be3ce8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:09.007876 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b09abf9-cae5-411a-96dc-9d6e65be3ce8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:56:09.008205 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b09abf9-cae5-411a-96dc-9d6e65be3ce8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:09.009876 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b09abf9-cae5-411a-96dc-9d6e65be3ce8 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:09.021875 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1420/2822] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:56:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:56:09.026122 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6240e312-0b56-4a7c-af57-d6424f2d26e7 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:09.035718 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6240e312-0b56-4a7c-af57-d6424f2d26e7 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:09.040406 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4edc78-f9ae-4ee2-b9fd-02abeb3a04eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:09.041054 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4edc78-f9ae-4ee2-b9fd-02abeb3a04eb admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:09.041372 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4edc78-f9ae-4ee2-b9fd-02abeb3a04eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:09.041653 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4edc78-f9ae-4ee2-b9fd-02abeb3a04eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:09.042419 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4edc78-f9ae-4ee2-b9fd-02abeb3a04eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:09.044311 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4edc78-f9ae-4ee2-b9fd-02abeb3a04eb admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:09.068546 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1403/2823] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:56:09 2022] POST /identity/v3/projects => generated 373 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:56:09.072632 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b09abf9-cae5-411a-96dc-9d6e65be3ce8 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:09.083239 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b09abf9-cae5-411a-96dc-9d6e65be3ce8 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:09.140056 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13643a99-0810-4061-9e41-b21b324e4c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:09.140729 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13643a99-0810-4061-9e41-b21b324e4c9b admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:09.141040 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13643a99-0810-4061-9e41-b21b324e4c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:09.141350 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13643a99-0810-4061-9e41-b21b324e4c9b admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:09.142076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13643a99-0810-4061-9e41-b21b324e4c9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:09.144087 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13643a99-0810-4061-9e41-b21b324e4c9b admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:09.164531 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1421/2824] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:56:09 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:56:09.168679 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4edc78-f9ae-4ee2-b9fd-02abeb3a04eb admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:09.179637 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4edc78-f9ae-4ee2-b9fd-02abeb3a04eb admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:09.184884 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c902001-00da-4166-9194-b84c6c7b01c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:09.185593 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c902001-00da-4166-9194-b84c6c7b01c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:09.185883 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c902001-00da-4166-9194-b84c6c7b01c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:09.186163 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c902001-00da-4166-9194-b84c6c7b01c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:09.186894 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c902001-00da-4166-9194-b84c6c7b01c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:09.188492 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c902001-00da-4166-9194-b84c6c7b01c7 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:09.202859 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1404/2825] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:56:09.203933 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13643a99-0810-4061-9e41-b21b324e4c9b admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:09.213892 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13643a99-0810-4061-9e41-b21b324e4c9b admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:09.218717 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e2a84-c503-4b53-a577-4de7bf8acc00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:09.219480 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e2a84-c503-4b53-a577-4de7bf8acc00 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:09.219771 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e2a84-c503-4b53-a577-4de7bf8acc00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:09.220053 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e2a84-c503-4b53-a577-4de7bf8acc00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:09.220824 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354e2a84-c503-4b53-a577-4de7bf8acc00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:09.222481 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354e2a84-c503-4b53-a577-4de7bf8acc00 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:09.237186 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1422/2826] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:56:09.238637 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c902001-00da-4166-9194-b84c6c7b01c7 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:09.250212 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c902001-00da-4166-9194-b84c6c7b01c7 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:09.258264 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a9753b-e3ac-4534-a43d-337f5d1b1b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:09.259247 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a9753b-e3ac-4534-a43d-337f5d1b1b1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:09.259612 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a9753b-e3ac-4534-a43d-337f5d1b1b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:09.259971 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a9753b-e3ac-4534-a43d-337f5d1b1b1a admin admin] PATH_INFO: `/v3/projects/06c1851ebcfe47a6bbbc177f1856a6cf/users/551de1461cd748cf8fd86d70a48360aa/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:09.281906 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a9753b-e3ac-4534-a43d-337f5d1b1b1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=06c1851ebcfe47a6bbbc177f1856a6cf, user_id=551de1461cd748cf8fd86d70a48360aa, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:09.286865 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a9753b-e3ac-4534-a43d-337f5d1b1b1a admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:09.306868 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1405/2827] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:56:09 2022] PUT /identity/v3/projects/06c1851ebcfe47a6bbbc177f1856a6cf/users/551de1461cd748cf8fd86d70a48360aa/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:56:09.318699 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49acee7-e0b7-4941-815d-79d80bd03ccb None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:09.318996 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49acee7-e0b7-4941-815d-79d80bd03ccb None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:09.319310 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49acee7-e0b7-4941-815d-79d80bd03ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:09.348393 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-d49acee7-e0b7-4941-815d-79d80bd03ccb None None] MFA Rules not processed for user `551de1461cd748cf8fd86d70a48360aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:56:09.349670 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d49acee7-e0b7-4941-815d-79d80bd03ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:56:09.394404 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:56:09.417706 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1423/2828] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:56:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:56:09.422336 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a9753b-e3ac-4534-a43d-337f5d1b1b1a admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:09.469801 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:56:09.488542 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a9753b-e3ac-4534-a43d-337f5d1b1b1a admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:09.498257 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55a1ee-3ff8-4c90-8017-fe40bdf17ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:09.498985 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55a1ee-3ff8-4c90-8017-fe40bdf17ea5 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:09.499275 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55a1ee-3ff8-4c90-8017-fe40bdf17ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:09.499557 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55a1ee-3ff8-4c90-8017-fe40bdf17ea5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:09.500263 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b55a1ee-3ff8-4c90-8017-fe40bdf17ea5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:09.502267 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b55a1ee-3ff8-4c90-8017-fe40bdf17ea5 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:09.528262 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1406/2829] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:56:09 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:56:09.532348 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49acee7-e0b7-4941-815d-79d80bd03ccb None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:09.543487 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49acee7-e0b7-4941-815d-79d80bd03ccb None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:09.601645 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c1523-b1f3-4289-a1d9-029d607dd75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:09.602321 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c1523-b1f3-4289-a1d9-029d607dd75d admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:09.602608 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c1523-b1f3-4289-a1d9-029d607dd75d admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:09.602889 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c1523-b1f3-4289-a1d9-029d607dd75d admin admin] PATH_INFO: `/v3/users` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:09.603593 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265c1523-b1f3-4289-a1d9-029d607dd75d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:09.606067 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265c1523-b1f3-4289-a1d9-029d607dd75d admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:09.626351 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1424/2830] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:56:09 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:56:09.630372 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55a1ee-3ff8-4c90-8017-fe40bdf17ea5 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:09.640390 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55a1ee-3ff8-4c90-8017-fe40bdf17ea5 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:09.644984 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b595eafb-8cee-4347-8067-08b8e5c45046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:09.645689 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b595eafb-8cee-4347-8067-08b8e5c45046 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:09.645972 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b595eafb-8cee-4347-8067-08b8e5c45046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:09.646248 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b595eafb-8cee-4347-8067-08b8e5c45046 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:09.646969 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b595eafb-8cee-4347-8067-08b8e5c45046 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:09.648672 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b595eafb-8cee-4347-8067-08b8e5c45046 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:09.658915 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1407/2831] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:56:09.662786 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c1523-b1f3-4289-a1d9-029d607dd75d admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:09.673238 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c1523-b1f3-4289-a1d9-029d607dd75d admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:09.677997 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38121c3e-52b4-42b1-b093-c6f93f67d096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:09.678870 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38121c3e-52b4-42b1-b093-c6f93f67d096 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:09.679159 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38121c3e-52b4-42b1-b093-c6f93f67d096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:09.679440 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38121c3e-52b4-42b1-b093-c6f93f67d096 admin admin] PATH_INFO: `/v3/projects/c585029bffdb4bd495682e97c421fc09/users/e427d636be8b44feaf22313c28ad1c81/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:09.695742 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38121c3e-52b4-42b1-b093-c6f93f67d096 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c585029bffdb4bd495682e97c421fc09, user_id=e427d636be8b44feaf22313c28ad1c81, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:09.698465 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38121c3e-52b4-42b1-b093-c6f93f67d096 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:09.711799 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1425/2832] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:56:09 2022] PUT /identity/v3/projects/c585029bffdb4bd495682e97c421fc09/users/e427d636be8b44feaf22313c28ad1c81/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:56:09.717322 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b595eafb-8cee-4347-8067-08b8e5c45046 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:09.726242 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b595eafb-8cee-4347-8067-08b8e5c45046 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:09.781658 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a35952-c827-4b67-baa7-3390b87ea96d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:09.782324 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a35952-c827-4b67-baa7-3390b87ea96d admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:09.782614 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a35952-c827-4b67-baa7-3390b87ea96d admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:09.782896 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a35952-c827-4b67-baa7-3390b87ea96d admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:09.783627 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a35952-c827-4b67-baa7-3390b87ea96d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:09.785287 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a35952-c827-4b67-baa7-3390b87ea96d admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:09.795497 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1408/2833] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:56:09.799845 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38121c3e-52b4-42b1-b093-c6f93f67d096 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:09.811717 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38121c3e-52b4-42b1-b093-c6f93f67d096 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:09.816324 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc508b9-4736-4fba-9505-8594e146cac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:09.817232 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc508b9-4736-4fba-9505-8594e146cac4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:09.817520 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc508b9-4736-4fba-9505-8594e146cac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:09.817797 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc508b9-4736-4fba-9505-8594e146cac4 admin admin] PATH_INFO: `/v3/projects/c585029bffdb4bd495682e97c421fc09/users/e427d636be8b44feaf22313c28ad1c81/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:09.819750 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc508b9-4736-4fba-9505-8594e146cac4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c585029bffdb4bd495682e97c421fc09, user_id=e427d636be8b44feaf22313c28ad1c81, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:09.822433 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc508b9-4736-4fba-9505-8594e146cac4 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:09.832325 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1426/2834] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:56:09 2022] PUT /identity/v3/projects/c585029bffdb4bd495682e97c421fc09/users/e427d636be8b44feaf22313c28ad1c81/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:56:09.842608 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8f691-478d-4236-8905-b993391beb30 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:09.842906 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8f691-478d-4236-8905-b993391beb30 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:09.843221 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8f691-478d-4236-8905-b993391beb30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:09.875915 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-3df8f691-478d-4236-8905-b993391beb30 None None] MFA Rules not processed for user `e427d636be8b44feaf22313c28ad1c81`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:56:09.877195 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-3df8f691-478d-4236-8905-b993391beb30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:56:09.945366 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1409/2835] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:56:09 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 16:56:09.955965 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc508b9-4736-4fba-9505-8594e146cac4 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:09.958007 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-efc508b9-4736-4fba-9505-8594e146cac4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:56:10.014327 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc508b9-4736-4fba-9505-8594e146cac4 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:10.017873 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e61de4a-b128-45a9-8fef-1ed0f7defb50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:10.018509 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61de4a-b128-45a9-8fef-1ed0f7defb50 service cinder] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:10.018790 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61de4a-b128-45a9-8fef-1ed0f7defb50 service cinder] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:10.019067 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61de4a-b128-45a9-8fef-1ed0f7defb50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:10.064715 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:56:10.074952 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e61de4a-b128-45a9-8fef-1ed0f7defb50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:10.076476 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e61de4a-b128-45a9-8fef-1ed0f7defb50 service cinder] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:10.085319 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1427/2836] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:56:09 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:56:12.585029 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df8f691-478d-4236-8905-b993391beb30 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:12.626756 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:56:12.648635 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df8f691-478d-4236-8905-b993391beb30 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:12.653174 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb38cb6-afc0-4c1b-8395-0ea30d995a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:12.654106 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb38cb6-afc0-4c1b-8395-0ea30d995a80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:12.654388 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb38cb6-afc0-4c1b-8395-0ea30d995a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:12.654670 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb38cb6-afc0-4c1b-8395-0ea30d995a80 admin admin] PATH_INFO: `/v3/users/551de1461cd748cf8fd86d70a48360aa` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:12.655865 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb38cb6-afc0-4c1b-8395-0ea30d995a80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=551de1461cd748cf8fd86d70a48360aa)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:12.657835 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb38cb6-afc0-4c1b-8395-0ea30d995a80 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:12.702961 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-2bb38cb6-afc0-4c1b-8395-0ea30d995a80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '551de1461cd748cf8fd86d70a48360aa'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:12.703597 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-2bb38cb6-afc0-4c1b-8395-0ea30d995a80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '551de1461cd748cf8fd86d70a48360aa'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:12.710598 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-2bb38cb6-afc0-4c1b-8395-0ea30d995a80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '551de1461cd748cf8fd86d70a48360aa'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:12.722246 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-2bb38cb6-afc0-4c1b-8395-0ea30d995a80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '551de1461cd748cf8fd86d70a48360aa'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:12.722901 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-2bb38cb6-afc0-4c1b-8395-0ea30d995a80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '551de1461cd748cf8fd86d70a48360aa'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:12.735236 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1410/2837] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:56:12 2022] DELETE /identity/v3/users/551de1461cd748cf8fd86d70a48360aa => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:56:12.739654 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fecff7c-577b-4caf-8fe4-e7ebb5055ee5 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:12.740143 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fecff7c-577b-4caf-8fe4-e7ebb5055ee5 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:12.740258 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fecff7c-577b-4caf-8fe4-e7ebb5055ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:12.757912 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-1fecff7c-577b-4caf-8fe4-e7ebb5055ee5 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:56:12.759121 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-1fecff7c-577b-4caf-8fe4-e7ebb5055ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:56:12.818246 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1428/2838] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:56:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:56:12.828064 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb38cb6-afc0-4c1b-8395-0ea30d995a80 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:12.829530 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-2bb38cb6-afc0-4c1b-8395-0ea30d995a80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:56:12.888882 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb38cb6-afc0-4c1b-8395-0ea30d995a80 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:12.895029 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404f0fb4-c8f4-4f59-bb84-ae8eaddbd7c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:12.895808 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f0fb4-c8f4-4f59-bb84-ae8eaddbd7c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:12.895971 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f0fb4-c8f4-4f59-bb84-ae8eaddbd7c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:12.896252 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f0fb4-c8f4-4f59-bb84-ae8eaddbd7c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:12.906815 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404f0fb4-c8f4-4f59-bb84-ae8eaddbd7c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:12.907927 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404f0fb4-c8f4-4f59-bb84-ae8eaddbd7c9 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:12.929491 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1411/2839] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:56:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:56:13.201115 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fecff7c-577b-4caf-8fe4-e7ebb5055ee5 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:13.202732 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-1fecff7c-577b-4caf-8fe4-e7ebb5055ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:56:13.216143 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fecff7c-577b-4caf-8fe4-e7ebb5055ee5 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:13.221119 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96622a21-5992-4530-9cbd-0ecdf741ae94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:13.222012 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96622a21-5992-4530-9cbd-0ecdf741ae94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:13.222827 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96622a21-5992-4530-9cbd-0ecdf741ae94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:13.223149 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96622a21-5992-4530-9cbd-0ecdf741ae94 admin admin] PATH_INFO: `/v3/projects/06c1851ebcfe47a6bbbc177f1856a6cf` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:13.224790 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96622a21-5992-4530-9cbd-0ecdf741ae94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06c1851ebcfe47a6bbbc177f1856a6cf)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:13.226752 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96622a21-5992-4530-9cbd-0ecdf741ae94 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:13.297428 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-96622a21-5992-4530-9cbd-0ecdf741ae94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06c1851ebcfe47a6bbbc177f1856a6cf', 'request_id': 'req-96622a21-5992-4530-9cbd-0ecdf741ae94'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:13.302913 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-96622a21-5992-4530-9cbd-0ecdf741ae94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06c1851ebcfe47a6bbbc177f1856a6cf', 'request_id': 'req-96622a21-5992-4530-9cbd-0ecdf741ae94'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:13.309602 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-96622a21-5992-4530-9cbd-0ecdf741ae94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06c1851ebcfe47a6bbbc177f1856a6cf', 'request_id': 'req-96622a21-5992-4530-9cbd-0ecdf741ae94'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:13.321728 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-96622a21-5992-4530-9cbd-0ecdf741ae94 admin admin] The token cache is being invalidate because project 06c1851ebcfe47a6bbbc177f1856a6cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:56:13.321934 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-96622a21-5992-4530-9cbd-0ecdf741ae94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:13.322556 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-96622a21-5992-4530-9cbd-0ecdf741ae94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:13.324168 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1429/2840] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:56:13 2022] DELETE /identity/v3/projects/06c1851ebcfe47a6bbbc177f1856a6cf => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:56:13.328090 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404f0fb4-c8f4-4f59-bb84-ae8eaddbd7c9 service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:13.337831 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404f0fb4-c8f4-4f59-bb84-ae8eaddbd7c9 service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:13.397057 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ac8ff4-b241-4b26-944b-ad0e3405dafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:13.397954 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac8ff4-b241-4b26-944b-ad0e3405dafd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:13.398236 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac8ff4-b241-4b26-944b-ad0e3405dafd admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:13.398510 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac8ff4-b241-4b26-944b-ad0e3405dafd admin admin] PATH_INFO: `/v3/users/e427d636be8b44feaf22313c28ad1c81` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:13.399682 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ac8ff4-b241-4b26-944b-ad0e3405dafd admin admin] RBAC: Authorizing `identity:delete_user(user_id=e427d636be8b44feaf22313c28ad1c81)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:13.401670 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ac8ff4-b241-4b26-944b-ad0e3405dafd admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:13.403081 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:56:13.448212 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-99ac8ff4-b241-4b26-944b-ad0e3405dafd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e427d636be8b44feaf22313c28ad1c81'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:13.448870 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-99ac8ff4-b241-4b26-944b-ad0e3405dafd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e427d636be8b44feaf22313c28ad1c81'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:13.455836 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-99ac8ff4-b241-4b26-944b-ad0e3405dafd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e427d636be8b44feaf22313c28ad1c81'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:13.467475 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-99ac8ff4-b241-4b26-944b-ad0e3405dafd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e427d636be8b44feaf22313c28ad1c81'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:13.468106 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-99ac8ff4-b241-4b26-944b-ad0e3405dafd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e427d636be8b44feaf22313c28ad1c81'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:13.483694 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1412/2841] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:56:13 2022] DELETE /identity/v3/users/e427d636be8b44feaf22313c28ad1c81 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:56:13.748490 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96622a21-5992-4530-9cbd-0ecdf741ae94 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:13.750378 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-96622a21-5992-4530-9cbd-0ecdf741ae94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:56:13.768903 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96622a21-5992-4530-9cbd-0ecdf741ae94 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:13.831443 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0765013-1735-4779-9a4f-af9102159f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:13.832249 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0765013-1735-4779-9a4f-af9102159f6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:13.832536 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0765013-1735-4779-9a4f-af9102159f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:13.832823 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0765013-1735-4779-9a4f-af9102159f6e admin admin] PATH_INFO: `/v3/projects/c585029bffdb4bd495682e97c421fc09` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:13.834073 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0765013-1735-4779-9a4f-af9102159f6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c585029bffdb4bd495682e97c421fc09)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:13.835995 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0765013-1735-4779-9a4f-af9102159f6e admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:13.851583 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:56:13.888738 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a0765013-1735-4779-9a4f-af9102159f6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c585029bffdb4bd495682e97c421fc09', 'request_id': 'req-a0765013-1735-4779-9a4f-af9102159f6e'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:13.895297 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a0765013-1735-4779-9a4f-af9102159f6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c585029bffdb4bd495682e97c421fc09', 'request_id': 'req-a0765013-1735-4779-9a4f-af9102159f6e'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:13.902181 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a0765013-1735-4779-9a4f-af9102159f6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c585029bffdb4bd495682e97c421fc09', 'request_id': 'req-a0765013-1735-4779-9a4f-af9102159f6e'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:13.914074 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a0765013-1735-4779-9a4f-af9102159f6e admin admin] The token cache is being invalidate because project c585029bffdb4bd495682e97c421fc09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:56:13.914367 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a0765013-1735-4779-9a4f-af9102159f6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:13.914993 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a0765013-1735-4779-9a4f-af9102159f6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:13.916552 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1430/2842] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:56:13 2022] DELETE /identity/v3/projects/c585029bffdb4bd495682e97c421fc09 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:56:13.922721 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea6d66-c412-490b-9f77-615e3805ef52 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:13.923208 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea6d66-c412-490b-9f77-615e3805ef52 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:13.923313 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea6d66-c412-490b-9f77-615e3805ef52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:13.941505 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-8eea6d66-c412-490b-9f77-615e3805ef52 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:56:13.942724 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-8eea6d66-c412-490b-9f77-615e3805ef52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:56:14.001396 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1413/2843] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:56:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:56:14.010286 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25143100-4f18-4859-92b8-5c588c9d5ae2 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:14.011093 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25143100-4f18-4859-92b8-5c588c9d5ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:14.011366 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25143100-4f18-4859-92b8-5c588c9d5ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:14.031036 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-25143100-4f18-4859-92b8-5c588c9d5ae2 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:56:14.032244 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-25143100-4f18-4859-92b8-5c588c9d5ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:56:14.040129 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1431/2844] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:56:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:56:14.044318 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eea6d66-c412-490b-9f77-615e3805ef52 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:14.054085 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eea6d66-c412-490b-9f77-615e3805ef52 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:14.058866 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5681fd-329a-4950-b987-e07a0f6b891e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:14.059535 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5681fd-329a-4950-b987-e07a0f6b891e admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:14.059822 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5681fd-329a-4950-b987-e07a0f6b891e admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:14.060100 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5681fd-329a-4950-b987-e07a0f6b891e admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:14.060903 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5681fd-329a-4950-b987-e07a0f6b891e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:56:14.061249 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5681fd-329a-4950-b987-e07a0f6b891e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:14.062858 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5681fd-329a-4950-b987-e07a0f6b891e admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:14.078441 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1414/2845] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:56:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:56:14.079688 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25143100-4f18-4859-92b8-5c588c9d5ae2 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:14.089535 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25143100-4f18-4859-92b8-5c588c9d5ae2 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:14.095072 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19760967-e232-480c-9df2-a2645253a9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:14.095741 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19760967-e232-480c-9df2-a2645253a9a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:14.096034 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19760967-e232-480c-9df2-a2645253a9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:14.096345 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19760967-e232-480c-9df2-a2645253a9a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:14.097018 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19760967-e232-480c-9df2-a2645253a9a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:14.098993 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19760967-e232-480c-9df2-a2645253a9a3 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:14.128879 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1432/2846] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:56:14 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:56:14.130512 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5681fd-329a-4950-b987-e07a0f6b891e admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:14.141139 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5681fd-329a-4950-b987-e07a0f6b891e admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:14.198099 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21401918-615f-49b4-a1ce-37e3c6570fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:14.199288 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21401918-615f-49b4-a1ce-37e3c6570fd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:14.199576 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21401918-615f-49b4-a1ce-37e3c6570fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:14.199858 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21401918-615f-49b4-a1ce-37e3c6570fd4 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:14.200568 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21401918-615f-49b4-a1ce-37e3c6570fd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:14.202510 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21401918-615f-49b4-a1ce-37e3c6570fd4 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:14.212203 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:56:14.228078 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1415/2847] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:56:14 2022] POST /identity/v3/users => generated 284 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:56:14.231347 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19760967-e232-480c-9df2-a2645253a9a3 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:14.245320 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19760967-e232-480c-9df2-a2645253a9a3 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:14.250406 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c192f3a6-e0fa-4be4-b326-8d0d52e7d285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:14.251636 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c192f3a6-e0fa-4be4-b326-8d0d52e7d285 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:14.252246 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c192f3a6-e0fa-4be4-b326-8d0d52e7d285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:14.252510 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c192f3a6-e0fa-4be4-b326-8d0d52e7d285 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:14.253277 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c192f3a6-e0fa-4be4-b326-8d0d52e7d285 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:14.254890 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c192f3a6-e0fa-4be4-b326-8d0d52e7d285 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:14.264909 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21401918-615f-49b4-a1ce-37e3c6570fd4 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:14.266646 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-21401918-615f-49b4-a1ce-37e3c6570fd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:56:14.269601 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1433/2848] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:56:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:56:14.274032 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c192f3a6-e0fa-4be4-b326-8d0d52e7d285 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:14.287227 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c192f3a6-e0fa-4be4-b326-8d0d52e7d285 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:14.294412 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6473ee-7173-4ba5-90f4-35933dcc8592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:14.295285 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6473ee-7173-4ba5-90f4-35933dcc8592 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:14.295575 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6473ee-7173-4ba5-90f4-35933dcc8592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:14.295877 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6473ee-7173-4ba5-90f4-35933dcc8592 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:14.296638 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6473ee-7173-4ba5-90f4-35933dcc8592 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:14.299354 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6473ee-7173-4ba5-90f4-35933dcc8592 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:14.313498 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1434/2849] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:56:14 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:56:14.316101 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6473ee-7173-4ba5-90f4-35933dcc8592 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:14.328704 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6473ee-7173-4ba5-90f4-35933dcc8592 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:14.334061 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d3f2a6-6a29-430f-b52a-80c4d9b818de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:14.334970 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3f2a6-6a29-430f-b52a-80c4d9b818de admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:14.335401 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3f2a6-6a29-430f-b52a-80c4d9b818de admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:14.335754 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3f2a6-6a29-430f-b52a-80c4d9b818de admin admin] PATH_INFO: `/v3/projects/c758aee0d6424991bd076491dba519d7/users/9672a8797ef34824bd7ea5de10fd7b3f/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:14.352334 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d3f2a6-6a29-430f-b52a-80c4d9b818de admin admin] RBAC: Authorizing `identity:create_grant(project_id=c758aee0d6424991bd076491dba519d7, user_id=9672a8797ef34824bd7ea5de10fd7b3f, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:14.356634 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d3f2a6-6a29-430f-b52a-80c4d9b818de admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:14.371635 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1435/2850] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:56:14 2022] PUT /identity/v3/projects/c758aee0d6424991bd076491dba519d7/users/9672a8797ef34824bd7ea5de10fd7b3f/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:56:14.382187 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1b1b4-6eee-43b6-8524-ec078bd8ac3b None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:14.382688 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1b1b4-6eee-43b6-8524-ec078bd8ac3b None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:14.383098 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1b1b4-6eee-43b6-8524-ec078bd8ac3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:14.399578 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21401918-615f-49b4-a1ce-37e3c6570fd4 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:14.414918 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-51d1b1b4-6eee-43b6-8524-ec078bd8ac3b None None] MFA Rules not processed for user `9672a8797ef34824bd7ea5de10fd7b3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:56:14.416262 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-51d1b1b4-6eee-43b6-8524-ec078bd8ac3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:56:14.471676 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb646d-124a-40f6-b8dc-6a8f2202f874 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e158e5c87f4954aeb297f42dabdc16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:14.472409 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb646d-124a-40f6-b8dc-6a8f2202f874 service placement] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:14.472729 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb646d-124a-40f6-b8dc-6a8f2202f874 service placement] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:14.472972 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb646d-124a-40f6-b8dc-6a8f2202f874 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:14.475099 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-44bb646d-124a-40f6-b8dc-6a8f2202f874 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:56:14.494259 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bb646d-124a-40f6-b8dc-6a8f2202f874 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:14.497335 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bb646d-124a-40f6-b8dc-6a8f2202f874 service placement] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:14.508629 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1436/2851] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:56:14 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:56:14.509016 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1416/2852] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:56:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 16:56:14.523076 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d1b1b4-6eee-43b6-8524-ec078bd8ac3b None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:14.524598 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-51d1b1b4-6eee-43b6-8524-ec078bd8ac3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:56:14.544198 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:56:14.587000 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d1b1b4-6eee-43b6-8524-ec078bd8ac3b None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:14.590889 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e2709-dc4a-4828-bd93-a9abab72d30b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57a5d7151478434f86be085bcc487f0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:14.591591 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e2709-dc4a-4828-bd93-a9abab72d30b service glance] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:14.591883 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e2709-dc4a-4828-bd93-a9abab72d30b service glance] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:14.592166 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e2709-dc4a-4828-bd93-a9abab72d30b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:14.601059 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9e2709-dc4a-4828-bd93-a9abab72d30b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:14.602652 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9e2709-dc4a-4828-bd93-a9abab72d30b service glance] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:14.612004 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1437/2853] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:56:14 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:56:24.504027 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=69 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:56:24.607485 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=18 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:56:27.486733 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb646d-124a-40f6-b8dc-6a8f2202f874 service placement] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:27.488116 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-44bb646d-124a-40f6-b8dc-6a8f2202f874 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:56:27.551164 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb646d-124a-40f6-b8dc-6a8f2202f874 service placement] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:27.556146 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72334f4b-095f-4e73-8bac-7df8728cb06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:27.557007 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72334f4b-095f-4e73-8bac-7df8728cb06d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:27.557326 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72334f4b-095f-4e73-8bac-7df8728cb06d admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:27.557603 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72334f4b-095f-4e73-8bac-7df8728cb06d admin admin] PATH_INFO: `/v3/users/bfac38790d0b4a288e91ca2b22f03477` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:27.558774 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72334f4b-095f-4e73-8bac-7df8728cb06d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfac38790d0b4a288e91ca2b22f03477)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:27.561354 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72334f4b-095f-4e73-8bac-7df8728cb06d admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:27.612098 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-72334f4b-095f-4e73-8bac-7df8728cb06d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfac38790d0b4a288e91ca2b22f03477'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:27.612748 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-72334f4b-095f-4e73-8bac-7df8728cb06d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfac38790d0b4a288e91ca2b22f03477'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:27.621417 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-72334f4b-095f-4e73-8bac-7df8728cb06d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfac38790d0b4a288e91ca2b22f03477'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:27.634328 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-72334f4b-095f-4e73-8bac-7df8728cb06d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfac38790d0b4a288e91ca2b22f03477'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:27.635024 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-72334f4b-095f-4e73-8bac-7df8728cb06d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfac38790d0b4a288e91ca2b22f03477'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:27.647196 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1417/2854] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:56:27 2022] DELETE /identity/v3/users/bfac38790d0b4a288e91ca2b22f03477 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:56:27.651671 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365b536-9e8a-4c96-97cf-f76f80ae6972 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:27.652140 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365b536-9e8a-4c96-97cf-f76f80ae6972 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:27.652263 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365b536-9e8a-4c96-97cf-f76f80ae6972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:27.669199 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-7365b536-9e8a-4c96-97cf-f76f80ae6972 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:56:27.670909 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-7365b536-9e8a-4c96-97cf-f76f80ae6972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:56:27.728410 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1438/2855] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:56:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:56:27.738438 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72334f4b-095f-4e73-8bac-7df8728cb06d admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:27.739834 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-72334f4b-095f-4e73-8bac-7df8728cb06d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:56:27.796133 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72334f4b-095f-4e73-8bac-7df8728cb06d admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:27.799830 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8588d2e2-7241-4511-bf07-4ee66cae623a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:27.800581 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8588d2e2-7241-4511-bf07-4ee66cae623a service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:27.801544 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8588d2e2-7241-4511-bf07-4ee66cae623a service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:27.801864 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8588d2e2-7241-4511-bf07-4ee66cae623a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:27.812031 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8588d2e2-7241-4511-bf07-4ee66cae623a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:27.813540 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8588d2e2-7241-4511-bf07-4ee66cae623a service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:27.823096 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1418/2856] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:56:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:56:28.096721 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7365b536-9e8a-4c96-97cf-f76f80ae6972 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:28.098172 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-7365b536-9e8a-4c96-97cf-f76f80ae6972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:56:28.112414 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7365b536-9e8a-4c96-97cf-f76f80ae6972 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:28.117922 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0537c7d-fc14-429c-8622-9fc5934c6ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:28.118961 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0537c7d-fc14-429c-8622-9fc5934c6ac3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:28.119474 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0537c7d-fc14-429c-8622-9fc5934c6ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:28.119888 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0537c7d-fc14-429c-8622-9fc5934c6ac3 admin admin] PATH_INFO: `/v3/projects/1eec3a9139624be69542440b311ee39d` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:28.121952 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0537c7d-fc14-429c-8622-9fc5934c6ac3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1eec3a9139624be69542440b311ee39d)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:28.124451 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0537c7d-fc14-429c-8622-9fc5934c6ac3 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:28.172704 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a0537c7d-fc14-429c-8622-9fc5934c6ac3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1eec3a9139624be69542440b311ee39d', 'request_id': 'req-a0537c7d-fc14-429c-8622-9fc5934c6ac3'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:28.178132 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a0537c7d-fc14-429c-8622-9fc5934c6ac3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1eec3a9139624be69542440b311ee39d', 'request_id': 'req-a0537c7d-fc14-429c-8622-9fc5934c6ac3'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:28.184192 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a0537c7d-fc14-429c-8622-9fc5934c6ac3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1eec3a9139624be69542440b311ee39d', 'request_id': 'req-a0537c7d-fc14-429c-8622-9fc5934c6ac3'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:28.196172 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a0537c7d-fc14-429c-8622-9fc5934c6ac3 admin admin] The token cache is being invalidate because project 1eec3a9139624be69542440b311ee39d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:56:28.196460 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a0537c7d-fc14-429c-8622-9fc5934c6ac3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:28.197087 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a0537c7d-fc14-429c-8622-9fc5934c6ac3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:28.198648 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1439/2857] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:56:28 2022] DELETE /identity/v3/projects/1eec3a9139624be69542440b311ee39d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:56:37.818826 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:56:38.193014 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:56:59.283388 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8588d2e2-7241-4511-bf07-4ee66cae623a service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:59.284804 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-8588d2e2-7241-4511-bf07-4ee66cae623a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:56:59.298247 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8588d2e2-7241-4511-bf07-4ee66cae623a service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:59.361318 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8f50a1-ffc5-438d-84e1-55e5d2d739ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:59.362311 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f50a1-ffc5-438d-84e1-55e5d2d739ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:59.362478 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f50a1-ffc5-438d-84e1-55e5d2d739ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:59.362757 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f50a1-ffc5-438d-84e1-55e5d2d739ae admin admin] PATH_INFO: `/v3/users/9672a8797ef34824bd7ea5de10fd7b3f` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:59.363966 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8f50a1-ffc5-438d-84e1-55e5d2d739ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=9672a8797ef34824bd7ea5de10fd7b3f)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:59.365911 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8f50a1-ffc5-438d-84e1-55e5d2d739ae admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:59.409932 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-aa8f50a1-ffc5-438d-84e1-55e5d2d739ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9672a8797ef34824bd7ea5de10fd7b3f'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:59.410572 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-aa8f50a1-ffc5-438d-84e1-55e5d2d739ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9672a8797ef34824bd7ea5de10fd7b3f'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:59.417613 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-aa8f50a1-ffc5-438d-84e1-55e5d2d739ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9672a8797ef34824bd7ea5de10fd7b3f'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:59.430329 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-aa8f50a1-ffc5-438d-84e1-55e5d2d739ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9672a8797ef34824bd7ea5de10fd7b3f'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:59.431013 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-aa8f50a1-ffc5-438d-84e1-55e5d2d739ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9672a8797ef34824bd7ea5de10fd7b3f'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:59.443335 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1419/2858] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:56:59 2022] DELETE /identity/v3/users/9672a8797ef34824bd7ea5de10fd7b3f => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:56:59.447887 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aeb494-c240-4c78-aa4b-29c2189a2301 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:59.448374 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aeb494-c240-4c78-aa4b-29c2189a2301 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:59.448482 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aeb494-c240-4c78-aa4b-29c2189a2301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:59.465940 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-d5aeb494-c240-4c78-aa4b-29c2189a2301 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:56:59.467598 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d5aeb494-c240-4c78-aa4b-29c2189a2301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:56:59.526255 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1440/2859] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:56:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:56:59.536545 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8f50a1-ffc5-438d-84e1-55e5d2d739ae admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:59.538018 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-aa8f50a1-ffc5-438d-84e1-55e5d2d739ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:56:59.594578 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8f50a1-ffc5-438d-84e1-55e5d2d739ae admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:59.598302 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b2103c-170b-443d-9de4-e5f1001f30ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:59.598954 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b2103c-170b-443d-9de4-e5f1001f30ca service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:59.599240 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b2103c-170b-443d-9de4-e5f1001f30ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:59.599521 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b2103c-170b-443d-9de4-e5f1001f30ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:59.609707 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b2103c-170b-443d-9de4-e5f1001f30ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:59.611150 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b2103c-170b-443d-9de4-e5f1001f30ca service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:59.620834 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1420/2860] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:56:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:56:59.884259 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aeb494-c240-4c78-aa4b-29c2189a2301 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:56:59.885712 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d5aeb494-c240-4c78-aa4b-29c2189a2301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:56:59.898345 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aeb494-c240-4c78-aa4b-29c2189a2301 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:56:59.903142 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89acdb3b-cf3c-46a9-97d7-efc712734348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:56:59.904010 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89acdb3b-cf3c-46a9-97d7-efc712734348 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:59.904294 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89acdb3b-cf3c-46a9-97d7-efc712734348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:59.905239 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89acdb3b-cf3c-46a9-97d7-efc712734348 admin admin] PATH_INFO: `/v3/projects/c758aee0d6424991bd076491dba519d7` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:56:59.906445 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89acdb3b-cf3c-46a9-97d7-efc712734348 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c758aee0d6424991bd076491dba519d7)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:56:59.908377 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89acdb3b-cf3c-46a9-97d7-efc712734348 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:56:59.959259 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-89acdb3b-cf3c-46a9-97d7-efc712734348 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c758aee0d6424991bd076491dba519d7', 'request_id': 'req-89acdb3b-cf3c-46a9-97d7-efc712734348'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:59.964669 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-89acdb3b-cf3c-46a9-97d7-efc712734348 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c758aee0d6424991bd076491dba519d7', 'request_id': 'req-89acdb3b-cf3c-46a9-97d7-efc712734348'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:59.971421 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-89acdb3b-cf3c-46a9-97d7-efc712734348 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c758aee0d6424991bd076491dba519d7', 'request_id': 'req-89acdb3b-cf3c-46a9-97d7-efc712734348'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:59.983192 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-89acdb3b-cf3c-46a9-97d7-efc712734348 admin admin] The token cache is being invalidate because project c758aee0d6424991bd076491dba519d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:56:59.983477 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-89acdb3b-cf3c-46a9-97d7-efc712734348 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:59.984138 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-89acdb3b-cf3c-46a9-97d7-efc712734348 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:56:59.985786 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1441/2861] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:56:59 2022] DELETE /identity/v3/projects/c758aee0d6424991bd076491dba519d7 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 16:56:59.990661 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd604b-2b0e-48fa-91ab-6eb2cb397f23 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:56:59.990959 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd604b-2b0e-48fa-91ab-6eb2cb397f23 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:56:59.991246 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd604b-2b0e-48fa-91ab-6eb2cb397f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:57:00.008622 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-1bdd604b-2b0e-48fa-91ab-6eb2cb397f23 None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:57:00.009841 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-1bdd604b-2b0e-48fa-91ab-6eb2cb397f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:57:00.026830 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:57:00.073932 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1421/2862] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:56:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:57:00.082247 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5151201-070e-476f-87cb-6872ea482feb None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:57:00.082582 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5151201-070e-476f-87cb-6872ea482feb None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:57:00.082848 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5151201-070e-476f-87cb-6872ea482feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:57:00.102299 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-a5151201-070e-476f-87cb-6872ea482feb None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:57:00.103545 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-a5151201-070e-476f-87cb-6872ea482feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:57:00.111371 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1442/2863] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:57:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:57:00.115554 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdd604b-2b0e-48fa-91ab-6eb2cb397f23 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:57:00.125942 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdd604b-2b0e-48fa-91ab-6eb2cb397f23 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:57:00.130502 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5344d005-a613-4979-8476-92614e026cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:57:00.131154 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344d005-a613-4979-8476-92614e026cc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:57:00.131439 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344d005-a613-4979-8476-92614e026cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:57:00.131715 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344d005-a613-4979-8476-92614e026cc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:57:00.132442 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5344d005-a613-4979-8476-92614e026cc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 16:57:00.132758 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5344d005-a613-4979-8476-92614e026cc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:57:00.134394 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5344d005-a613-4979-8476-92614e026cc1 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:57:00.146370 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1422/2864] 10.222.0.36 () {58 vars in 1152 bytes} [Thu May 19 16:57:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 16:57:00.150362 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5151201-070e-476f-87cb-6872ea482feb None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:57:00.159692 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5151201-070e-476f-87cb-6872ea482feb None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:57:00.164485 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2867884f-90c8-44a0-b954-bc3dff6fb264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:57:00.165127 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867884f-90c8-44a0-b954-bc3dff6fb264 admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:57:00.165447 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867884f-90c8-44a0-b954-bc3dff6fb264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:57:00.165725 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867884f-90c8-44a0-b954-bc3dff6fb264 admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:57:00.166414 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2867884f-90c8-44a0-b954-bc3dff6fb264 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:57:00.168316 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2867884f-90c8-44a0-b954-bc3dff6fb264 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:57:00.193814 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1443/2865] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:57:00 2022] POST /identity/v3/projects => generated 379 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:57:00.198028 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5344d005-a613-4979-8476-92614e026cc1 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:57:00.208547 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5344d005-a613-4979-8476-92614e026cc1 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:57:00.264332 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e2f20b-f161-41df-863b-c06556c90a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:57:00.265007 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2f20b-f161-41df-863b-c06556c90a76 admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:57:00.265331 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2f20b-f161-41df-863b-c06556c90a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:57:00.265616 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2f20b-f161-41df-863b-c06556c90a76 admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:57:00.266303 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e2f20b-f161-41df-863b-c06556c90a76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:57:00.268213 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e2f20b-f161-41df-863b-c06556c90a76 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:57:00.287195 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1423/2866] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:57:00 2022] POST /identity/v3/users => generated 290 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:57:00.292337 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2867884f-90c8-44a0-b954-bc3dff6fb264 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:57:00.302651 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2867884f-90c8-44a0-b954-bc3dff6fb264 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:57:00.307291 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69916040-a2f8-4c64-94b2-5f191c9e0720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:57:00.307948 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69916040-a2f8-4c64-94b2-5f191c9e0720 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:57:00.308235 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69916040-a2f8-4c64-94b2-5f191c9e0720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:57:00.308519 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69916040-a2f8-4c64-94b2-5f191c9e0720 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:57:00.309292 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69916040-a2f8-4c64-94b2-5f191c9e0720 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:57:00.311613 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69916040-a2f8-4c64-94b2-5f191c9e0720 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:57:00.321888 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1444/2867] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:57:00.325933 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e2f20b-f161-41df-863b-c06556c90a76 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:57:00.336310 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e2f20b-f161-41df-863b-c06556c90a76 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:57:00.340937 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7f83a0-2b13-4133-b3bc-7aa1a170da37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:57:00.341622 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f83a0-2b13-4133-b3bc-7aa1a170da37 admin admin] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:57:00.341910 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f83a0-2b13-4133-b3bc-7aa1a170da37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:57:00.342190 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f83a0-2b13-4133-b3bc-7aa1a170da37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:57:00.342917 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7f83a0-2b13-4133-b3bc-7aa1a170da37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:57:00.344500 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7f83a0-2b13-4133-b3bc-7aa1a170da37 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:57:00.354214 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1424/2868] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:57:00.358174 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69916040-a2f8-4c64-94b2-5f191c9e0720 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:57:00.368081 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69916040-a2f8-4c64-94b2-5f191c9e0720 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:57:00.372935 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e1ae56-83e8-46bf-9c2a-dde638762336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:57:00.374523 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1ae56-83e8-46bf-9c2a-dde638762336 admin admin] REQUEST_METHOD: `PUT` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:57:00.374812 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1ae56-83e8-46bf-9c2a-dde638762336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:57:00.375092 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1ae56-83e8-46bf-9c2a-dde638762336 admin admin] PATH_INFO: `/v3/projects/dc95250983234edeb99b67ff4f084416/users/aaf1ce6ace6e47d6a170ce17afa5397f/roles/0e98fd94a6d24e46b9fe09a22a7df04a` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:57:00.389859 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e1ae56-83e8-46bf-9c2a-dde638762336 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc95250983234edeb99b67ff4f084416, user_id=aaf1ce6ace6e47d6a170ce17afa5397f, role_id=0e98fd94a6d24e46b9fe09a22a7df04a)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:57:00.392510 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e1ae56-83e8-46bf-9c2a-dde638762336 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:57:00.404925 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1445/2869] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:57:00 2022] PUT /identity/v3/projects/dc95250983234edeb99b67ff4f084416/users/aaf1ce6ace6e47d6a170ce17afa5397f/roles/0e98fd94a6d24e46b9fe09a22a7df04a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:57:00.410024 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce28875-ced4-40cc-87aa-27fdb5145e2f None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:57:00.410326 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce28875-ced4-40cc-87aa-27fdb5145e2f None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:57:00.410615 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce28875-ced4-40cc-87aa-27fdb5145e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:57:00.428902 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-7ce28875-ced4-40cc-87aa-27fdb5145e2f None None] MFA Rules not processed for user `ac4bfc4d6198473c8d280a14b2c6036f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:57:00.430220 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-7ce28875-ced4-40cc-87aa-27fdb5145e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:57:00.487467 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1425/2870] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:57:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:57:00.497300 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e1ae56-83e8-46bf-9c2a-dde638762336 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:57:00.499612 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-92e1ae56-83e8-46bf-9c2a-dde638762336 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:57:00.521422 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:57:00.557040 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e1ae56-83e8-46bf-9c2a-dde638762336 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:57:00.561286 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a57421-a4f1-46b5-940d-6cfd77cda501 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:57:00.561932 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a57421-a4f1-46b5-940d-6cfd77cda501 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:57:00.562219 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a57421-a4f1-46b5-940d-6cfd77cda501 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:57:00.562499 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a57421-a4f1-46b5-940d-6cfd77cda501 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:57:00.571349 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a57421-a4f1-46b5-940d-6cfd77cda501 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:57:00.572793 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a57421-a4f1-46b5-940d-6cfd77cda501 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:57:00.582274 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1446/2871] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:57:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:57:04.570387 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc5e82-0902-4347-bfe2-55bf5c4e87e4 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:57:04.570936 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc5e82-0902-4347-bfe2-55bf5c4e87e4 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:57:04.570936 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc5e82-0902-4347-bfe2-55bf5c4e87e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:57:04.599898 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-dedc5e82-0902-4347-bfe2-55bf5c4e87e4 None None] MFA Rules not processed for user `aaf1ce6ace6e47d6a170ce17afa5397f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:57:04.601309 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-dedc5e82-0902-4347-bfe2-55bf5c4e87e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:57:04.624041 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:57:04.707674 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1426/2872] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:57:04 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:57:04.712209 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a57421-a4f1-46b5-940d-6cfd77cda501 service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:57:04.722783 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a57421-a4f1-46b5-940d-6cfd77cda501 service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:57:04.727587 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896c9cc5-8c34-4a43-a6da-f43339509e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:57:04.728328 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896c9cc5-8c34-4a43-a6da-f43339509e2a admin admin] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:57:04.728680 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896c9cc5-8c34-4a43-a6da-f43339509e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:57:04.729022 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896c9cc5-8c34-4a43-a6da-f43339509e2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:57:04.729833 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896c9cc5-8c34-4a43-a6da-f43339509e2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:57:04.731848 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896c9cc5-8c34-4a43-a6da-f43339509e2a admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:57:04.762511 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1447/2873] 10.222.0.36 () {60 vars in 1152 bytes} [Thu May 19 16:57:04 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 16:57:04.763742 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc5e82-0902-4347-bfe2-55bf5c4e87e4 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:57:04.775527 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc5e82-0902-4347-bfe2-55bf5c4e87e4 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:57:04.845034 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc367a7-5cb2-4edc-88cb-3d2755e088ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:57:04.845760 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc367a7-5cb2-4edc-88cb-3d2755e088ac admin admin] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:57:04.846055 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc367a7-5cb2-4edc-88cb-3d2755e088ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:57:04.846343 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc367a7-5cb2-4edc-88cb-3d2755e088ac admin admin] PATH_INFO: `/v3/users` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:57:04.847053 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc367a7-5cb2-4edc-88cb-3d2755e088ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:57:04.848993 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc367a7-5cb2-4edc-88cb-3d2755e088ac admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:57:04.872133 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1427/2874] 10.222.0.36 () {60 vars in 1143 bytes} [Thu May 19 16:57:04 2022] POST /identity/v3/users => generated 295 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 16:57:04.876282 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896c9cc5-8c34-4a43-a6da-f43339509e2a admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:57:04.887057 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896c9cc5-8c34-4a43-a6da-f43339509e2a admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:57:04.891980 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b67bf6-ed97-41f2-a733-41ae40544022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:57:04.892664 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b67bf6-ed97-41f2-a733-41ae40544022 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:57:04.892975 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b67bf6-ed97-41f2-a733-41ae40544022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:57:04.893289 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b67bf6-ed97-41f2-a733-41ae40544022 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:57:04.894084 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b67bf6-ed97-41f2-a733-41ae40544022 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:57:04.895655 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b67bf6-ed97-41f2-a733-41ae40544022 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:57:04.908020 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1448/2875] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 16:57:04.911973 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc367a7-5cb2-4edc-88cb-3d2755e088ac admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:57:04.922894 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc367a7-5cb2-4edc-88cb-3d2755e088ac admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:57:04.927753 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967db51a-cd73-48b3-a329-54eb58df0230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:57:04.928703 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967db51a-cd73-48b3-a329-54eb58df0230 admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:57:04.928980 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967db51a-cd73-48b3-a329-54eb58df0230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:57:04.929285 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967db51a-cd73-48b3-a329-54eb58df0230 admin admin] PATH_INFO: `/v3/projects/50835675114e4bf2881eb7d02fbbd676/users/4ee1068d412946788a338b4cf8aab2d7/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:57:04.947556 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967db51a-cd73-48b3-a329-54eb58df0230 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50835675114e4bf2881eb7d02fbbd676, user_id=4ee1068d412946788a338b4cf8aab2d7, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:57:04.950570 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967db51a-cd73-48b3-a329-54eb58df0230 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:57:04.967792 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1428/2876] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:57:04 2022] PUT /identity/v3/projects/50835675114e4bf2881eb7d02fbbd676/users/4ee1068d412946788a338b4cf8aab2d7/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:57:04.971976 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b67bf6-ed97-41f2-a733-41ae40544022 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:57:04.992349 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b67bf6-ed97-41f2-a733-41ae40544022 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:57:05.060949 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f9fab5-35a0-40d1-af56-c256255f6f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:57:05.061673 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f9fab5-35a0-40d1-af56-c256255f6f69 admin admin] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:57:05.061954 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f9fab5-35a0-40d1-af56-c256255f6f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:57:05.062234 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f9fab5-35a0-40d1-af56-c256255f6f69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:57:05.062988 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f9fab5-35a0-40d1-af56-c256255f6f69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:57:05.064618 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f9fab5-35a0-40d1-af56-c256255f6f69 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:57:05.081638 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1449/2877] 10.222.0.36 () {58 vars in 1121 bytes} [Thu May 19 16:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 16:57:05.082832 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967db51a-cd73-48b3-a329-54eb58df0230 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:57:05.114298 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967db51a-cd73-48b3-a329-54eb58df0230 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:57:05.125593 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7460302-33b1-45e4-9b65-56ed2eb731ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:57:05.126616 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7460302-33b1-45e4-9b65-56ed2eb731ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:57:05.126981 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7460302-33b1-45e4-9b65-56ed2eb731ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:57:05.127318 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7460302-33b1-45e4-9b65-56ed2eb731ab admin admin] PATH_INFO: `/v3/projects/50835675114e4bf2881eb7d02fbbd676/users/4ee1068d412946788a338b4cf8aab2d7/roles/e2915fe109244a07a9be96bd2b219fdd` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:57:05.129491 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7460302-33b1-45e4-9b65-56ed2eb731ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=50835675114e4bf2881eb7d02fbbd676, user_id=4ee1068d412946788a338b4cf8aab2d7, role_id=e2915fe109244a07a9be96bd2b219fdd)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:57:05.132202 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7460302-33b1-45e4-9b65-56ed2eb731ab admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:57:05.143527 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1429/2878] 10.222.0.36 () {60 vars in 1482 bytes} [Thu May 19 16:57:05 2022] PUT /identity/v3/projects/50835675114e4bf2881eb7d02fbbd676/users/4ee1068d412946788a338b4cf8aab2d7/roles/e2915fe109244a07a9be96bd2b219fdd => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:57:09.757978 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1867ca-811f-427f-8888-441cec776b4d None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:57:09.758493 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1867ca-811f-427f-8888-441cec776b4d None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:57:09.758609 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1867ca-811f-427f-8888-441cec776b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:57:09.794047 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-0e1867ca-811f-427f-8888-441cec776b4d None None] MFA Rules not processed for user `4ee1068d412946788a338b4cf8aab2d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:57:09.795758 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-0e1867ca-811f-427f-8888-441cec776b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:57:09.892038 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1450/2879] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:57:09 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:57:09.905365 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7460302-33b1-45e4-9b65-56ed2eb731ab admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:57:09.908279 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-e7460302-33b1-45e4-9b65-56ed2eb731ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:57:09.991182 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7460302-33b1-45e4-9b65-56ed2eb731ab admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:57:09.997050 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ea503-8e73-4a8b-9d53-9d36accc8936 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:57:09.997736 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ea503-8e73-4a8b-9d53-9d36accc8936 service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:57:09.998529 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ea503-8e73-4a8b-9d53-9d36accc8936 service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:57:09.998806 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ea503-8e73-4a8b-9d53-9d36accc8936 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:57:10.005206 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:57:10.071180 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36ea503-8e73-4a8b-9d53-9d36accc8936 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:57:10.072857 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36ea503-8e73-4a8b-9d53-9d36accc8936 service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:57:10.084291 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1430/2880] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:57:09 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:57:19.884926 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=94,keystone:INSERT=1 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:57:20.079417 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=24 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:57:32.667075 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52068864-62c3-454a-956a-bb7d0cbd6829 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:57:32.667596 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52068864-62c3-454a-956a-bb7d0cbd6829 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:57:32.667708 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52068864-62c3-454a-956a-bb7d0cbd6829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:57:32.684680 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-52068864-62c3-454a-956a-bb7d0cbd6829 None None] MFA Rules not processed for user `aaf1ce6ace6e47d6a170ce17afa5397f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 16:57:32.685915 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-52068864-62c3-454a-956a-bb7d0cbd6829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:57:32.692677 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1451/2881] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 16:57:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 16:57:42.679434 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:57:54.483416 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ea503-8e73-4a8b-9d53-9d36accc8936 service nova] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:57:54.536647 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ea503-8e73-4a8b-9d53-9d36accc8936 service nova] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:57:54.541013 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1ecee4-6478-442d-94bb-5154f0a28cbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:57:54.541706 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ecee4-6478-442d-94bb-5154f0a28cbd service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:57:54.541994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ecee4-6478-442d-94bb-5154f0a28cbd service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:57:54.542274 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ecee4-6478-442d-94bb-5154f0a28cbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:57:54.552239 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1ecee4-6478-442d-94bb-5154f0a28cbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:57:54.553710 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1ecee4-6478-442d-94bb-5154f0a28cbd service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:57:54.563598 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1431/2882] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:57:54 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:58:00.503289 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52068864-62c3-454a-956a-bb7d0cbd6829 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:58:00.513117 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52068864-62c3-454a-956a-bb7d0cbd6829 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:58:00.516707 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5b5279-164e-4164-85f8-f664d3df6b84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:58:00.517450 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5b5279-164e-4164-85f8-f664d3df6b84 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:58:00.517744 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5b5279-164e-4164-85f8-f664d3df6b84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:58:00.518027 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5b5279-164e-4164-85f8-f664d3df6b84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:58:00.519846 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-6b5b5279-164e-4164-85f8-f664d3df6b84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:58:00.576690 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5b5279-164e-4164-85f8-f664d3df6b84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:58:00.578203 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5b5279-164e-4164-85f8-f664d3df6b84 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:58:00.587122 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1452/2883] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:58:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:58:03.744573 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1ecee4-6478-442d-94bb-5154f0a28cbd service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:58:03.770280 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1ecee4-6478-442d-94bb-5154f0a28cbd service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:58:03.775902 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcbe53e-70c8-4ff3-af09-07cd4e44330f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:58:03.776553 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbe53e-70c8-4ff3-af09-07cd4e44330f service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:58:03.776841 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbe53e-70c8-4ff3-af09-07cd4e44330f service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:58:03.777180 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbe53e-70c8-4ff3-af09-07cd4e44330f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:58:03.779082 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-bbcbe53e-70c8-4ff3-af09-07cd4e44330f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:58:03.805401 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcbe53e-70c8-4ff3-af09-07cd4e44330f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:58:03.807403 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcbe53e-70c8-4ff3-af09-07cd4e44330f service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:58:03.819614 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1432/2884] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:58:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:58:05.554838 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5b5279-164e-4164-85f8-f664d3df6b84 service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:58:05.556267 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-6b5b5279-164e-4164-85f8-f664d3df6b84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:58:05.567160 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5b5279-164e-4164-85f8-f664d3df6b84 service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:58:05.570978 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164ad833-70e2-4aba-b16a-4c9fa6a6f3d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:58:05.571569 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ad833-70e2-4aba-b16a-4c9fa6a6f3d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:58:05.571861 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ad833-70e2-4aba-b16a-4c9fa6a6f3d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:58:05.572150 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ad833-70e2-4aba-b16a-4c9fa6a6f3d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:58:05.573921 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-164ad833-70e2-4aba-b16a-4c9fa6a6f3d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:58:05.636779 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164ad833-70e2-4aba-b16a-4c9fa6a6f3d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:58:05.638380 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164ad833-70e2-4aba-b16a-4c9fa6a6f3d3 service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:58:05.648403 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1453/2885] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:58:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:58:11.643281 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcbe53e-70c8-4ff3-af09-07cd4e44330f service nova] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:58:11.644830 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-bbcbe53e-70c8-4ff3-af09-07cd4e44330f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:58:11.659683 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcbe53e-70c8-4ff3-af09-07cd4e44330f service nova] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:58:11.664375 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc457f7-a9da-45d9-9085-e7c982c7d762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:58:11.665280 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc457f7-a9da-45d9-9085-e7c982c7d762 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:58:11.665567 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc457f7-a9da-45d9-9085-e7c982c7d762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:58:11.665874 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc457f7-a9da-45d9-9085-e7c982c7d762 admin admin] PATH_INFO: `/v3/users/708dbabdda134292a8259982db495e1f` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:58:11.667057 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc457f7-a9da-45d9-9085-e7c982c7d762 admin admin] RBAC: Authorizing `identity:delete_user(user_id=708dbabdda134292a8259982db495e1f)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:58:11.668996 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc457f7-a9da-45d9-9085-e7c982c7d762 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:58:11.715805 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-7dc457f7-a9da-45d9-9085-e7c982c7d762 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '708dbabdda134292a8259982db495e1f'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:58:11.716461 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-7dc457f7-a9da-45d9-9085-e7c982c7d762 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '708dbabdda134292a8259982db495e1f'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:58:11.723502 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-7dc457f7-a9da-45d9-9085-e7c982c7d762 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '708dbabdda134292a8259982db495e1f'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:58:11.735779 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-7dc457f7-a9da-45d9-9085-e7c982c7d762 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '708dbabdda134292a8259982db495e1f'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:58:11.736478 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-7dc457f7-a9da-45d9-9085-e7c982c7d762 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '708dbabdda134292a8259982db495e1f'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:58:11.748895 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1433/2886] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:58:11 2022] DELETE /identity/v3/users/708dbabdda134292a8259982db495e1f => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:58:11.919977 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164ad833-70e2-4aba-b16a-4c9fa6a6f3d3 service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:58:11.932761 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164ad833-70e2-4aba-b16a-4c9fa6a6f3d3 service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:58:11.990680 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3283a4a-4f4d-4640-8055-a18ee329dc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:58:11.991545 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3283a4a-4f4d-4640-8055-a18ee329dc70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:58:11.991837 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3283a4a-4f4d-4640-8055-a18ee329dc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:58:11.992119 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3283a4a-4f4d-4640-8055-a18ee329dc70 admin admin] PATH_INFO: `/v3/projects/f232c172df994d3d909d7999238b1935` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:58:11.993392 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3283a4a-4f4d-4640-8055-a18ee329dc70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f232c172df994d3d909d7999238b1935)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:58:11.995653 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3283a4a-4f4d-4640-8055-a18ee329dc70 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:58:12.033282 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:58:12.043865 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a3283a4a-4f4d-4640-8055-a18ee329dc70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f232c172df994d3d909d7999238b1935', 'request_id': 'req-a3283a4a-4f4d-4640-8055-a18ee329dc70'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:58:12.049906 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a3283a4a-4f4d-4640-8055-a18ee329dc70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f232c172df994d3d909d7999238b1935', 'request_id': 'req-a3283a4a-4f4d-4640-8055-a18ee329dc70'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:58:12.056043 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a3283a4a-4f4d-4640-8055-a18ee329dc70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f232c172df994d3d909d7999238b1935', 'request_id': 'req-a3283a4a-4f4d-4640-8055-a18ee329dc70'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:58:12.067810 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a3283a4a-4f4d-4640-8055-a18ee329dc70 admin admin] The token cache is being invalidate because project f232c172df994d3d909d7999238b1935 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:58:12.068093 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a3283a4a-4f4d-4640-8055-a18ee329dc70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:58:12.068806 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-a3283a4a-4f4d-4640-8055-a18ee329dc70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:58:12.070426 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1454/2887] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:58:11 2022] DELETE /identity/v3/projects/f232c172df994d3d909d7999238b1935 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:58:12.074507 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc457f7-a9da-45d9-9085-e7c982c7d762 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:58:12.076008 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-7dc457f7-a9da-45d9-9085-e7c982c7d762 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:58:12.089667 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc457f7-a9da-45d9-9085-e7c982c7d762 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:58:12.130822 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:58:12.149022 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5064e0-9163-446a-985b-4d1e7686af65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:58:12.149859 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5064e0-9163-446a-985b-4d1e7686af65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:58:12.150147 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5064e0-9163-446a-985b-4d1e7686af65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:58:12.150426 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5064e0-9163-446a-985b-4d1e7686af65 admin admin] PATH_INFO: `/v3/users/8e962e3b904248068c708d4d07c72a89` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:58:12.151622 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5064e0-9163-446a-985b-4d1e7686af65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e962e3b904248068c708d4d07c72a89)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:58:12.153565 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5064e0-9163-446a-985b-4d1e7686af65 admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:58:12.198642 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-1c5064e0-9163-446a-985b-4d1e7686af65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e962e3b904248068c708d4d07c72a89'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:58:12.199294 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-1c5064e0-9163-446a-985b-4d1e7686af65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e962e3b904248068c708d4d07c72a89'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:58:12.206932 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-1c5064e0-9163-446a-985b-4d1e7686af65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e962e3b904248068c708d4d07c72a89'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:58:12.219697 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-1c5064e0-9163-446a-985b-4d1e7686af65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e962e3b904248068c708d4d07c72a89'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:58:12.220337 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-1c5064e0-9163-446a-985b-4d1e7686af65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e962e3b904248068c708d4d07c72a89'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:58:12.232400 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1434/2888] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 16:58:12 2022] DELETE /identity/v3/users/8e962e3b904248068c708d4d07c72a89 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:58:12.413314 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3283a4a-4f4d-4640-8055-a18ee329dc70 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:58:12.414961 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-a3283a4a-4f4d-4640-8055-a18ee329dc70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:58:12.428432 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3283a4a-4f4d-4640-8055-a18ee329dc70 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:58:12.486106 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c9ed53-c732-4765-a009-335604a741be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:58:12.486961 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9ed53-c732-4765-a009-335604a741be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:58:12.487246 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9ed53-c732-4765-a009-335604a741be admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:58:12.487522 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9ed53-c732-4765-a009-335604a741be admin admin] PATH_INFO: `/v3/projects/34dfa2f0b34644ce84102414e307276e` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:58:12.488728 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c9ed53-c732-4765-a009-335604a741be admin admin] RBAC: Authorizing `identity:delete_project(project_id=34dfa2f0b34644ce84102414e307276e)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:58:12.490651 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c9ed53-c732-4765-a009-335604a741be admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:58:12.536266 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-e5c9ed53-c732-4765-a009-335604a741be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34dfa2f0b34644ce84102414e307276e', 'request_id': 'req-e5c9ed53-c732-4765-a009-335604a741be'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:58:12.541873 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-e5c9ed53-c732-4765-a009-335604a741be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34dfa2f0b34644ce84102414e307276e', 'request_id': 'req-e5c9ed53-c732-4765-a009-335604a741be'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:58:12.548292 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-e5c9ed53-c732-4765-a009-335604a741be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34dfa2f0b34644ce84102414e307276e', 'request_id': 'req-e5c9ed53-c732-4765-a009-335604a741be'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:58:12.560064 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-e5c9ed53-c732-4765-a009-335604a741be admin admin] The token cache is being invalidate because project 34dfa2f0b34644ce84102414e307276e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 16:58:12.560349 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-e5c9ed53-c732-4765-a009-335604a741be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:58:12.560968 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-e5c9ed53-c732-4765-a009-335604a741be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 16:58:12.563089 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1455/2889] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 16:58:12 2022] DELETE /identity/v3/projects/34dfa2f0b34644ce84102414e307276e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 16:58:22.228930 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=34,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:58:22.557214 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:DELETE=12,keystone:SELECT=62,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 16:59:21.010110 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5064e0-9163-446a-985b-4d1e7686af65 admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 16:59:21.011579 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-1c5064e0-9163-446a-985b-4d1e7686af65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:59:21.067741 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5064e0-9163-446a-985b-4d1e7686af65 admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 16:59:21.114545 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3237ec4f-4766-41a2-9fee-84147acc6a3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 16:59:21.115211 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237ec4f-4766-41a2-9fee-84147acc6a3b service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 16:59:21.115508 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237ec4f-4766-41a2-9fee-84147acc6a3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 16:59:21.115775 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237ec4f-4766-41a2-9fee-84147acc6a3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 16:59:21.118330 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-3237ec4f-4766-41a2-9fee-84147acc6a3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 16:59:21.128071 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3237ec4f-4766-41a2-9fee-84147acc6a3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 16:59:21.129546 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3237ec4f-4766-41a2-9fee-84147acc6a3b service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 16:59:21.137582 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1435/2890] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 16:59:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 16:59:31.133886 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=46 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 17:00:10.393684 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c9ed53-c732-4765-a009-335604a741be admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 17:00:10.395171 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e5c9ed53-c732-4765-a009-335604a741be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 17:00:10.408624 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c9ed53-c732-4765-a009-335604a741be admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 17:00:10.412234 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e96fc47-9d29-4d42-ab36-34f72c2eb4cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 17:00:10.412877 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e96fc47-9d29-4d42-ab36-34f72c2eb4cf service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 17:00:10.413197 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e96fc47-9d29-4d42-ab36-34f72c2eb4cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 17:00:10.413485 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e96fc47-9d29-4d42-ab36-34f72c2eb4cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 17:00:10.415252 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-4e96fc47-9d29-4d42-ab36-34f72c2eb4cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 17:00:10.489438 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e96fc47-9d29-4d42-ab36-34f72c2eb4cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 17:00:10.489438 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e96fc47-9d29-4d42-ab36-34f72c2eb4cf service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 17:00:10.503922 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1456/2891] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 17:00:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 17:00:13.941730 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80175ddf-c9f6-417e-a354-561acea41a86 None None] REQUEST_METHOD: `POST` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 17:00:13.942277 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80175ddf-c9f6-417e-a354-561acea41a86 None None] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 17:00:13.942277 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80175ddf-c9f6-417e-a354-561acea41a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 17:00:13.960992 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.auth.core [None req-80175ddf-c9f6-417e-a354-561acea41a86 None None] MFA Rules not processed for user `aaf1ce6ace6e47d6a170ce17afa5397f`. Rule list: `[]` (Enabled: `True`). {{(pid=80467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 17:00:13.962255 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-80175ddf-c9f6-417e-a354-561acea41a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 17:00:14.020755 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1436/2892] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 17:00:13 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 17:00:14.349880 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e96fc47-9d29-4d42-ab36-34f72c2eb4cf service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 17:00:14.359140 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e96fc47-9d29-4d42-ab36-34f72c2eb4cf service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 17:00:14.362721 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033daa31-df4c-46ba-80d5-de5ba750734e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 17:00:14.363378 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033daa31-df4c-46ba-80d5-de5ba750734e service neutron] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 17:00:14.363668 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033daa31-df4c-46ba-80d5-de5ba750734e service neutron] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 17:00:14.363946 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033daa31-df4c-46ba-80d5-de5ba750734e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 17:00:14.365879 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-033daa31-df4c-46ba-80d5-de5ba750734e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 17:00:14.375890 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033daa31-df4c-46ba-80d5-de5ba750734e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 17:00:14.377378 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033daa31-df4c-46ba-80d5-de5ba750734e service neutron] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 17:00:14.385446 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1457/2893] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 17:00:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 17:00:14.829552 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80175ddf-c9f6-417e-a354-561acea41a86 None None] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 17:00:14.831046 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-80175ddf-c9f6-417e-a354-561acea41a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 17:00:14.906957 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80175ddf-c9f6-417e-a354-561acea41a86 None None] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 17:00:14.911970 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbad9e7-6225-49e9-a763-e7be5675383d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e158e5c87f4954aeb297f42dabdc16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 17:00:14.912656 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbad9e7-6225-49e9-a763-e7be5675383d service placement] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 17:00:14.912943 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbad9e7-6225-49e9-a763-e7be5675383d service placement] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 17:00:14.913257 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbad9e7-6225-49e9-a763-e7be5675383d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 17:00:14.915062 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-0bbad9e7-6225-49e9-a763-e7be5675383d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 17:00:14.926379 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbad9e7-6225-49e9-a763-e7be5675383d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 17:00:14.927994 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbad9e7-6225-49e9-a763-e7be5675383d service placement] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 17:00:14.938129 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1437/2894] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 17:00:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 17:00:24.381744 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=40 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 17:00:24.934623 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=57 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 17:00:28.100109 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033daa31-df4c-46ba-80d5-de5ba750734e service neutron] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 17:00:28.101731 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-033daa31-df4c-46ba-80d5-de5ba750734e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 17:00:28.121176 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033daa31-df4c-46ba-80d5-de5ba750734e service neutron] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 17:00:28.126709 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916762b4-89b6-47c6-9626-2674f0252bb7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 17:00:28.127360 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916762b4-89b6-47c6-9626-2674f0252bb7 service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 17:00:28.127654 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916762b4-89b6-47c6-9626-2674f0252bb7 service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 17:00:28.127941 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916762b4-89b6-47c6-9626-2674f0252bb7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 17:00:28.129757 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-916762b4-89b6-47c6-9626-2674f0252bb7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 17:00:28.142636 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916762b4-89b6-47c6-9626-2674f0252bb7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 17:00:28.144184 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916762b4-89b6-47c6-9626-2674f0252bb7 service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 17:00:28.155003 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1458/2895] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 17:00:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 17:00:31.785801 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbad9e7-6225-49e9-a763-e7be5675383d service placement] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 17:00:31.798739 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbad9e7-6225-49e9-a763-e7be5675383d service placement] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 17:00:31.804193 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc533f8-3eae-4178-be22-346b9333082a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 17:00:31.804837 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc533f8-3eae-4178-be22-346b9333082a service nova] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 17:00:31.805119 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc533f8-3eae-4178-be22-346b9333082a service nova] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 17:00:31.805425 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc533f8-3eae-4178-be22-346b9333082a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 17:00:31.807199 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-9fc533f8-3eae-4178-be22-346b9333082a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 17:00:31.820142 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc533f8-3eae-4178-be22-346b9333082a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 17:00:31.822076 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc533f8-3eae-4178-be22-346b9333082a service nova] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 17:00:31.833020 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1438/2896] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 17:00:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 17:00:38.151060 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=6 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 17:00:41.828212 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=6 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 17:01:20.251241 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916762b4-89b6-47c6-9626-2674f0252bb7 service nova] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 17:01:20.252646 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-916762b4-89b6-47c6-9626-2674f0252bb7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 17:01:20.266704 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916762b4-89b6-47c6-9626-2674f0252bb7 service nova] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 17:01:20.271627 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7afb3e5-2374-439a-b048-adbc4b4c0660 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e158e5c87f4954aeb297f42dabdc16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 17:01:20.272905 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7afb3e5-2374-439a-b048-adbc4b4c0660 service placement] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 17:01:20.273366 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7afb3e5-2374-439a-b048-adbc4b4c0660 service placement] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 17:01:20.273660 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7afb3e5-2374-439a-b048-adbc4b4c0660 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 17:01:20.275338 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-e7afb3e5-2374-439a-b048-adbc4b4c0660 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 17:01:20.286550 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7afb3e5-2374-439a-b048-adbc4b4c0660 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 17:01:20.288120 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7afb3e5-2374-439a-b048-adbc4b4c0660 service placement] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 17:01:20.298030 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1459/2897] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 17:01:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 17:01:30.293899 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=6 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 17:02:20.601313 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc533f8-3eae-4178-be22-346b9333082a service nova] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 17:02:20.602723 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-9fc533f8-3eae-4178-be22-346b9333082a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 17:02:20.658747 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc533f8-3eae-4178-be22-346b9333082a service nova] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 17:02:20.662386 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85690217-65d9-481b-8e6d-3823c022a699 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc96d859a344042a4251a4104419855', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 17:02:20.663022 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85690217-65d9-481b-8e6d-3823c022a699 service cinder] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 17:02:20.663812 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85690217-65d9-481b-8e6d-3823c022a699 service cinder] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 17:02:20.664088 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85690217-65d9-481b-8e6d-3823c022a699 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 17:02:20.665870 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-85690217-65d9-481b-8e6d-3823c022a699 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 17:02:20.676207 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85690217-65d9-481b-8e6d-3823c022a699 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 17:02:20.677666 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85690217-65d9-481b-8e6d-3823c022a699 service cinder] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 17:02:20.687080 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1439/2898] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 17:02:20 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 17:02:30.683009 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=26 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 17:02:31.644110 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1cee4d-7fea-4990-bec0-6b1c511bca10 None None] REQUEST_METHOD: `POST` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 17:02:31.644677 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1cee4d-7fea-4990-bec0-6b1c511bca10 None None] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 17:02:31.644810 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1cee4d-7fea-4990-bec0-6b1c511bca10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 17:02:31.663277 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.auth.core [None req-7e1cee4d-7fea-4990-bec0-6b1c511bca10 None None] MFA Rules not processed for user `aaf1ce6ace6e47d6a170ce17afa5397f`. Rule list: `[]` (Enabled: `True`). {{(pid=80466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 17:02:31.665235 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-7e1cee4d-7fea-4990-bec0-6b1c511bca10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 17:02:31.672359 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1460/2899] 10.222.0.36 () {58 vars in 957 bytes} [Thu May 19 17:02:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 17:02:41.657633 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=3 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 17:04:26.022409 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85690217-65d9-481b-8e6d-3823c022a699 service cinder] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 17:04:26.023862 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-85690217-65d9-481b-8e6d-3823c022a699 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 17:04:26.063454 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85690217-65d9-481b-8e6d-3823c022a699 service cinder] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 17:04:26.067020 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2276735-8840-4220-8e73-2a802dab878f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 17:04:26.068294 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2276735-8840-4220-8e73-2a802dab878f service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 17:04:26.068583 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2276735-8840-4220-8e73-2a802dab878f service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 17:04:26.068862 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2276735-8840-4220-8e73-2a802dab878f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 17:04:26.076085 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2276735-8840-4220-8e73-2a802dab878f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 17:04:26.077545 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2276735-8840-4220-8e73-2a802dab878f service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 17:04:26.085502 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1440/2900] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 17:04:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 17:04:36.081845 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=19 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 17:04:43.874307 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1cee4d-7fea-4990-bec0-6b1c511bca10 None None] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 17:04:43.918056 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1cee4d-7fea-4990-bec0-6b1c511bca10 None None] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 17:04:43.923720 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caa6c61-f3df-4475-a77b-55d4c6fc418d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58bdeada00b74b7b8e186e6dc51e6dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 17:04:43.924370 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa6c61-f3df-4475-a77b-55d4c6fc418d service nova] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 17:04:43.924655 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa6c61-f3df-4475-a77b-55d4c6fc418d service nova] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 17:04:43.924932 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa6c61-f3df-4475-a77b-55d4c6fc418d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 17:04:43.926770 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-1caa6c61-f3df-4475-a77b-55d4c6fc418d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 17:04:43.939670 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caa6c61-f3df-4475-a77b-55d4c6fc418d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 17:04:43.941326 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caa6c61-f3df-4475-a77b-55d4c6fc418d service nova] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 17:04:43.956122 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1461/2901] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 17:04:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 17:04:53.845891 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2276735-8840-4220-8e73-2a802dab878f service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 17:04:53.866502 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2276735-8840-4220-8e73-2a802dab878f service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 17:04:53.870328 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae615c47-5553-4d78-83d9-57e9067abc66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33472643b65e4fe08fd5b3aa2c618133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 17:04:53.870975 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae615c47-5553-4d78-83d9-57e9067abc66 service neutron] REQUEST_METHOD: `GET` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 17:04:53.871263 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae615c47-5553-4d78-83d9-57e9067abc66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 17:04:53.871544 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae615c47-5553-4d78-83d9-57e9067abc66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 17:04:53.873478 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-ae615c47-5553-4d78-83d9-57e9067abc66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 17:04:53.948175 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=17 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 17:04:53.957528 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae615c47-5553-4d78-83d9-57e9067abc66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 17:04:53.958991 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae615c47-5553-4d78-83d9-57e9067abc66 service neutron] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 17:04:53.977963 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1441/2902] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 17:04:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 17:04:59.948525 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caa6c61-f3df-4475-a77b-55d4c6fc418d service nova] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 17:04:59.950034 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-1caa6c61-f3df-4475-a77b-55d4c6fc418d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 17:04:59.965023 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caa6c61-f3df-4475-a77b-55d4c6fc418d service nova] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 17:04:59.969756 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a3089-dfc5-4714-9020-1701cafc9f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 17:04:59.970684 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a3089-dfc5-4714-9020-1701cafc9f62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 17:04:59.970970 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a3089-dfc5-4714-9020-1701cafc9f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 17:04:59.971252 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a3089-dfc5-4714-9020-1701cafc9f62 admin admin] PATH_INFO: `/v3/users/aaf1ce6ace6e47d6a170ce17afa5397f` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 17:04:59.972448 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816a3089-dfc5-4714-9020-1701cafc9f62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aaf1ce6ace6e47d6a170ce17afa5397f)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 17:04:59.974954 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816a3089-dfc5-4714-9020-1701cafc9f62 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 17:05:00.022934 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-816a3089-dfc5-4714-9020-1701cafc9f62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aaf1ce6ace6e47d6a170ce17afa5397f'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 17:05:00.023588 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-816a3089-dfc5-4714-9020-1701cafc9f62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aaf1ce6ace6e47d6a170ce17afa5397f'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 17:05:00.024195 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-816a3089-dfc5-4714-9020-1701cafc9f62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aaf1ce6ace6e47d6a170ce17afa5397f'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 17:05:00.031991 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-816a3089-dfc5-4714-9020-1701cafc9f62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aaf1ce6ace6e47d6a170ce17afa5397f'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 17:05:00.044689 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-816a3089-dfc5-4714-9020-1701cafc9f62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aaf1ce6ace6e47d6a170ce17afa5397f'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 17:05:00.057498 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1462/2903] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 17:04:59 2022] DELETE /identity/v3/users/aaf1ce6ace6e47d6a170ce17afa5397f => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 17:05:00.241603 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae615c47-5553-4d78-83d9-57e9067abc66 service neutron] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 17:05:00.252307 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae615c47-5553-4d78-83d9-57e9067abc66 service neutron] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 17:05:00.310236 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0c9647-0140-429d-9e20-770f720a983b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 17:05:00.311109 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c9647-0140-429d-9e20-770f720a983b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 17:05:00.311434 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c9647-0140-429d-9e20-770f720a983b admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 17:05:00.311719 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c9647-0140-429d-9e20-770f720a983b admin admin] PATH_INFO: `/v3/projects/dc95250983234edeb99b67ff4f084416` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 17:05:00.312906 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0c9647-0140-429d-9e20-770f720a983b admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc95250983234edeb99b67ff4f084416)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 17:05:00.314882 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0c9647-0140-429d-9e20-770f720a983b admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 17:05:00.360056 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-6e0c9647-0140-429d-9e20-770f720a983b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc95250983234edeb99b67ff4f084416', 'request_id': 'req-6e0c9647-0140-429d-9e20-770f720a983b'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 17:05:00.365876 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-6e0c9647-0140-429d-9e20-770f720a983b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc95250983234edeb99b67ff4f084416', 'request_id': 'req-6e0c9647-0140-429d-9e20-770f720a983b'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 17:05:00.367349 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 17:05:00.374499 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-6e0c9647-0140-429d-9e20-770f720a983b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc95250983234edeb99b67ff4f084416', 'request_id': 'req-6e0c9647-0140-429d-9e20-770f720a983b'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 17:05:00.386377 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-6e0c9647-0140-429d-9e20-770f720a983b admin admin] The token cache is being invalidate because project dc95250983234edeb99b67ff4f084416 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 17:05:00.386666 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-6e0c9647-0140-429d-9e20-770f720a983b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 17:05:00.387282 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-6e0c9647-0140-429d-9e20-770f720a983b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 17:05:00.388818 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1442/2904] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 17:05:00 2022] DELETE /identity/v3/projects/dc95250983234edeb99b67ff4f084416 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 17:05:00.392782 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a3089-dfc5-4714-9020-1701cafc9f62 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 17:05:00.394201 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-816a3089-dfc5-4714-9020-1701cafc9f62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 17:05:00.407358 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a3089-dfc5-4714-9020-1701cafc9f62 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 17:05:00.465493 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689173c7-877f-4948-92c7-79693aaaaa02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 17:05:00.467158 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689173c7-877f-4948-92c7-79693aaaaa02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 17:05:00.467445 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689173c7-877f-4948-92c7-79693aaaaa02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 17:05:00.467724 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689173c7-877f-4948-92c7-79693aaaaa02 admin admin] PATH_INFO: `/v3/users/4ee1068d412946788a338b4cf8aab2d7` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 17:05:00.468928 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689173c7-877f-4948-92c7-79693aaaaa02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ee1068d412946788a338b4cf8aab2d7)` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 17:05:00.470921 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689173c7-877f-4948-92c7-79693aaaaa02 admin admin] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 17:05:00.517289 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-689173c7-877f-4948-92c7-79693aaaaa02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ee1068d412946788a338b4cf8aab2d7'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 17:05:00.517858 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-689173c7-877f-4948-92c7-79693aaaaa02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ee1068d412946788a338b4cf8aab2d7'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 17:05:00.518540 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-689173c7-877f-4948-92c7-79693aaaaa02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ee1068d412946788a338b4cf8aab2d7'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 17:05:00.525572 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-689173c7-877f-4948-92c7-79693aaaaa02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ee1068d412946788a338b4cf8aab2d7'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 17:05:00.537780 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.notifications [None req-689173c7-877f-4948-92c7-79693aaaaa02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ee1068d412946788a338b4cf8aab2d7'} {{(pid=80466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 17:05:00.539768 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 17:05:00.552036 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1463/2905] 10.222.0.36 () {58 vars in 1223 bytes} [Thu May 19 17:05:00 2022] DELETE /identity/v3/users/4ee1068d412946788a338b4cf8aab2d7 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 17:05:00.723593 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0c9647-0140-429d-9e20-770f720a983b admin admin] Authenticating user token {{(pid=80467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 17:05:00.725604 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.fernet_utils [None req-6e0c9647-0140-429d-9e20-770f720a983b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 17:05:00.738198 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0c9647-0140-429d-9e20-770f720a983b admin admin] Validating token access rules against request {{(pid=80467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 17:05:00.795672 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ace2294-0d26-4cca-a557-f56ae2259aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4bfc4d6198473c8d280a14b2c6036f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d5de9b2c50341a8b78fb369c7765787', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 17:05:00.796499 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace2294-0d26-4cca-a557-f56ae2259aff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 17:05:00.796786 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace2294-0d26-4cca-a557-f56ae2259aff admin admin] SCRIPT_NAME: `/identity` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 17:05:00.797066 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace2294-0d26-4cca-a557-f56ae2259aff admin admin] PATH_INFO: `/v3/projects/50835675114e4bf2881eb7d02fbbd676` {{(pid=80467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 17:05:00.798286 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ace2294-0d26-4cca-a557-f56ae2259aff admin admin] RBAC: Authorizing `identity:delete_project(project_id=50835675114e4bf2881eb7d02fbbd676)` {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 17:05:00.800193 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ace2294-0d26-4cca-a557-f56ae2259aff admin admin] RBAC: Authorization granted {{(pid=80467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 17:05:00.845197 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-9ace2294-0d26-4cca-a557-f56ae2259aff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50835675114e4bf2881eb7d02fbbd676', 'request_id': 'req-9ace2294-0d26-4cca-a557-f56ae2259aff'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 17:05:00.851523 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-9ace2294-0d26-4cca-a557-f56ae2259aff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50835675114e4bf2881eb7d02fbbd676', 'request_id': 'req-9ace2294-0d26-4cca-a557-f56ae2259aff'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 17:05:00.857549 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-9ace2294-0d26-4cca-a557-f56ae2259aff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50835675114e4bf2881eb7d02fbbd676', 'request_id': 'req-9ace2294-0d26-4cca-a557-f56ae2259aff'} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 17:05:00.870411 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-9ace2294-0d26-4cca-a557-f56ae2259aff admin admin] The token cache is being invalidate because project 50835675114e4bf2881eb7d02fbbd676 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 17:05:00.870702 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-9ace2294-0d26-4cca-a557-f56ae2259aff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 17:05:00.871324 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG keystone.notifications [None req-9ace2294-0d26-4cca-a557-f56ae2259aff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 17:05:00.872848 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: [pid: 80467|app: 0|req: 1443/2906] 10.222.0.36 () {58 vars in 1232 bytes} [Thu May 19 17:05:00 2022] DELETE /identity/v3/projects/50835675114e4bf2881eb7d02fbbd676 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 17:05:10.551234 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=4 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 17:05:10.867204 dv-cinder-842433-1-iscsi devstack@keystone.service[80467]: DEBUG dbcounter [-] [80467] Writing DB stats keystone:SELECT=55,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 17:06:26.233480 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689173c7-877f-4948-92c7-79693aaaaa02 admin admin] Authenticating user token {{(pid=80466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 17:06:26.234880 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-689173c7-877f-4948-92c7-79693aaaaa02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 17:06:26.325102 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689173c7-877f-4948-92c7-79693aaaaa02 admin admin] Validating token access rules against request {{(pid=80466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 17:06:26.383734 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337c769-a0b4-434b-b983-50dbe4f1006d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6e158e5c87f4954aeb297f42dabdc16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631c76794a124d6da22c626e590e4352', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 17:06:26.384381 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337c769-a0b4-434b-b983-50dbe4f1006d service placement] REQUEST_METHOD: `GET` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 17:06:26.384668 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337c769-a0b4-434b-b983-50dbe4f1006d service placement] SCRIPT_NAME: `/identity` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 17:06:26.384949 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337c769-a0b4-434b-b983-50dbe4f1006d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 17:06:26.387358 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.fernet_utils [None req-d337c769-a0b4-434b-b983-50dbe4f1006d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 17:06:26.398357 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d337c769-a0b4-434b-b983-50dbe4f1006d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 17:06:26.399954 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d337c769-a0b4-434b-b983-50dbe4f1006d service placement] RBAC: Authorization granted {{(pid=80466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 17:06:26.409764 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: [pid: 80466|app: 0|req: 1464/2907] 10.222.0.36 () {60 vars in 1365 bytes} [Thu May 19 17:06:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 17:06:36.405491 dv-cinder-842433-1-iscsi devstack@keystone.service[80466]: DEBUG dbcounter [-] [80466] Writing DB stats keystone:SELECT=67 {{(pid=80466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}